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Preface 


For  the  past  several  years,  the  Air  Force  has  been  using  data  from  the 
Individual  Contracting  Action  Report  form,  also  known  as  the 
DD350,  to  analyze  its  purchase  of  goods  and  services.  These  analyses 
support  Air  Force  efforts  to  develop  new  purchasing  and  supply  man¬ 
agement  strategies  for  important  categories  of  goods  and  services. 
Fiowever,  concerns  have  been  raised  about  whether  the  DD350  data 
are  accurate  enough  and  detailed  enough  to  conduct  such  spend 
analyses.  To  evaluate  the  suitability  of  these  data  for  analyzing  the 
purchase  of  goods  and  services,  RAND  Project  AIR  FORCE  col¬ 
lected  data  to  supplement  information  found  in  a  sample  of  Air  Force 
fiscal  year  2002  (FY02)  DD350  records.  This  monograph  describes 
the  survey  and  statistical  methods  used  in  this  study  as  well  as  the  re¬ 
searchers’  assessment  of  the  DD350  data,  which  is  based  on  an 
extrapolation  of  the  survey  results  to  the  entire  Air  Force  FY02 
database. 

This  research  was  part  of  a  broader  study,  “Supporting  Air  Force 
Procurement  Transformation  and  Laying  the  Groundwork  for  Serv¬ 
ices  Acquisition  Reform,”  sponsored  by  the  Air  Force  Deputy  Assis¬ 
tant  Secretary  for  Contracting  (SAF/AQC)  and  conducted  within  the 
Resource  Management  Program  of  RAND  Project  AIR  FORCE. 

This  report  is  designed  to  assist  Department  of  Defense  person¬ 
nel  involved  in  conducting  spend  analyses  and  implementing  com¬ 
mercial  purchasing  and  supply  management  practices.  As  such,  the 
authors  assume  that  the  reader  has  a  basic  understanding  of  commer- 
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cial  purchasing  and  supply  management  practices,  including  the  use 
of  spend  analyses  to  support  the  design  of  purchasing  strategies. 

For  the  past  decade,  RAND  Project  AIR  FORCE  has  been 
helping  the  Air  Force  to  reshape  its  sourcing  policies  and  practices. 
Related  RAND  reports  that  may  be  of  interest  include  the  following: 

•  Air  Force  Service  Procurement :  Approaches  for  Measurement  and 
Management ,  by  Laura  H.  Baldwin,  John  A.  Ausink,  and  Nancy 
Nicosia,  MG-299-AF,  2005. 

•  Air  Force  Procurement  Workforce  Transformation:  Lessons  from  the 
Commercial  Sector,  by  John  A.  Ausink,  Laura  H.  Baldwin,  and 
Christopher  Paul,  MG-214-AF,  2004. 

•  Organizational  Concepts  for  Purchasing  and  Supply  Management 
Implementation,  by  Lynne  M.  Leftwich,  James  Leftwich,  Nancy 
Y.  Moore,  and  Charles  Robert  Roll,  MG-1 16-AF,  2004. 

•  Measuring  Changes  in  Service  Costs  to  Meet  the  Requirements  of 
the  2002  National  Defense  Authorization  Act,  by  Chad  Shirley, 
John  A.  Ausink,  and  Laura  H.  Baldwin,  MR-1821-AF,  2004. 

•  Defining  Needs  and  Managing  Performance  of  Installation  Support 
Contracts:  Perspectives  from  the  Commercial  Sector,  by  Laura  H. 
Baldwin  and  Sarah  Hunter,  MR-1812-AF,  2004. 

•  Using  a  Spend  Analysis  to  Help  Identify  Prospective  Air  Force  Pur¬ 
chasing  and  Supply  Management  Initiatives:  Summary  of  Selected 
Findings,  by  Nancy  Y.  Moore,  Cynthia  Cook,  Clifford  Gram- 
mich,  and  Charles  Lindenblatt,  DB-434-AF,  2004. 

•  Implementing  Performance-Based  Services  Acquisition  (PBSA): 
Perspectives  from  an  Air  Logistics  Center  and  a  Product  Center,  by 
John  A.  Ausink,  Laura  H.  Baldwin,  Sarah  Hunter,  and  Chad 
Shirley,  DB-388-AF,  2002. 

•  Implementing  Best  Purchasing  and  Supply  Management  Practices: 
Lessons  from  Innovative  Commercial  Firms,  by  Nancy  Y.  Moore, 
Laura  H.  Baldwin,  Frank  A.  Camm,  and  Cynthia  R.  Cook,  DB- 
334-AF,  2002. 

•  Federal  Contract  Bundling:  A  Framework  for  Making  and  Justify¬ 
ing  Decisions  for  Purchased  Services,  by  Laura  H.  Baldwin,  Frank 
A.  Camm,  and  Nancy  Y.  Moore,  MR-1224-AF,  2001. 
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•  Performance-Based  Contracting  in  the  Air  Force:  A  Report  on  Expe¬ 
riences  in  the  Field,  by  John  A.  Ausink,  Frank  A.  Camm,  and 
Charles  Cannon,  DB-342-AF,  2001. 

•  Strategic  Sourcing:  Measuring  and  Managing  Performance,  by 
Laura  H.  Baldwin,  Frank  A.  Camm,  and  Nancy  Y.  Moore,  DB- 
287-AF,  2000. 

•  Incentives  to  Undertake  Sourcing  Studies  in  the  Air  Force,  by  Laura 
H.  Baldwin,  Frank  A.  Camm,  Edward  G.  Keating,  and  Ellen  M. 
Pint,  DB-240-AF,  1998. 

•  Strategic  Sourcing:  Theory  and  Evidence  from  Economics  and  Busi¬ 
ness  Management,  by  Ellen  M.  Pint  and  Laura  H.  Baldwin,  MR- 
865-AF,  1997. 

RAND  Project  AIR  FORCE 

RAND  Project  AIR  FORCE  (PAF),  a  division  of  the  RAND  Corpo¬ 
ration,  is  the  U.S.  Air  Force’s  federally  funded  research  and  develop¬ 
ment  center  for  studies  and  analyses.  PAF  provides  the  Air  Force  with 
independent  analyses  of  policy  alternatives  affecting  the  development, 
employment,  combat  readiness,  and  support  of  current  and  future 
aerospace  forces.  Research  is  performed  in  four  programs:  Aerospace 
Force  Development;  Manpower,  Personnel,  and  Training;  Resource 
Management;  and  Strategy  and  Doctrine. 

Additional  information  about  PAF  is  available  on  our  Web  site 
at  http://www.rand.org/paf. 
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Summary 


More  than  a  decade  ago,  commercial  firms  began  to  change  the  way 
they  purchase  goods  and  services  to  become  more  competitive  in  the 
marketplace.  Through  better  management  of  their  supply  bases  and 
supplier  relationships  and  more  sophisticated  purchasing  strategies, 
these  firms  have  improved  supplier  performance  and  reduced  the 
prices  of  their  purchases  for  various  categories  of  goods  and  services. 
A  primary  input  to  constructing  the  purchasing  and  supply- 
management  strategies  used  by  these  firms  is  what  is  commonly  called 
a  “spend  analysis”  of  a  firm’s  contract  expenditures  and  supply  base. 
This  analysis  can  be  used  to  identify  potential  targets  of  opportunity 
for  change,  which  in  turn  enables  firms  to  leverage  their  expenditures 
and  design  improved  strategies  for  interacting  with  suppliers. 

For  several  years,  the  Air  Force  has  been  working  to  incorporate 
widely  accepted  commercial  purchasing  and  supply  management 
practices  into  its  purchases  of  equipment  and  supplies.  These  efforts 
are  currently  being  expanded  to  include  the  purchase  of  services.  To 
assist  in  these  efforts,  the  Air  Force  Deputy  Assistant  Secretary  for 
Contracting  (SAF/AQC)  asked  RAND  Project  AIR  FORCE  to  con¬ 
duct  an  initial,  high-level  analysis  of  Air  Force  expenditures  to  deter¬ 
mine  how  much  the  Air  Force  spends  on  different  types  of  services. 

The  primary  source  of  data  on  Air  Force  contract  expenditures  is 
the  DD350  database.  The  DD350  database  contains  data  from  the 
Individual  Contracting  Action  Report  form  (also  known  as  the 
DD350  form).  The  DD350  database  provides  a  good  deal  of  infor¬ 
mation  on  contract  transactions  greater  than  $25,000,  but  it  was  de¬ 
signed  to  support  a  variety  of  compliance-oriented  analyses  (e.g., 
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analyses  of  purchases  from  small  businesses  or  minority-owned  busi¬ 
nesses),  rather  than  the  types  of  detailed  analyses  of  expenditures  that 
are  required  for  spend  analyses. 

In  particular,  the  DD350  form  does  not  allow  for  a  detailed  de¬ 
scription  of  the  various  types  of  goods  and  services  the  Air  Force  pur¬ 
chases.  The  Product  Service  Codes  (PSCs)  used  in  the  DD350  form 
to  describe  purchases  are  not  detailed  enough  to  capture  the  full  range 
of  goods  and  services  purchased,  and  contracting  officers  are  not 
trained  in  how  to  assign  PSCs  in  a  consistent  way.  Finally,  many  of 
the  most  important  DD350  data  fields  from  a  spend  analysis  perspec¬ 
tive  are  unaudited  (i.e.,  they  are  not  checked  for  completeness  and 
accuracy) . 

Given  these  reasons  for  concern  about  the  usefulness  of  the 
DD350  data  for  spend  analyses,  we  undertook  a  formal  assessment  of 
the  accuracy  and  completeness  of  key  data  fields  that  describe  Air 
Force  purchases.  The  possibility  that  purchases  of  services  may  be 
embedded  in  contract  actions  coded  as  purchases  of  goods  (and  vice 
versa)  led  us  to  examine  data  for  both  goods  and  services  contracts.1 
We  collected  supplemental  information  on  a  sample  of  306  contract 
actions  from  Air  Force  contracting  officers.  The  sample  was  weighted 
more  heavily  toward  larger-dollar-value  contract  actions  because  we 
hypothesized  that  those  actions  are  more  complex  than  lesser-value 
ones,  and  thus  the  information  in  the  DD350  database  on  larger 
transactions  may  be  less  reliable  than  the  information  on  smaller 
transactions. 

To  assess  the  accuracy  and  completeness  of  the  information  in 
the  DD350  database,  we  interviewed  the  contracting  officers  respon¬ 
sible  for  entering  the  data  on  selected  contract  actions  to  learn  more 
about  the  nature  of  each  purchase.  We  then  used  statistical  methods 
to  extrapolate  our  findings  from  the  sample  to  all  contract  actions  in 
the  Air  Force  fiscal  year  2002  (FY02)  DD350  database. 


1  A  single  contract  between  the  Air  Force  and  a  supplier  can  have  multiple  contract  actions 
associated  with  it.  Such  actions  may  represent  incremental  obligations  of  funds,  deobliga¬ 
tions,  orders  against  indefinite  delivery  and  indefinite  quantity  contracts,  and  other  contract 
modifications. 
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Table  S.l  summarizes  our  key  findings  on  the  accuracy  and  ade¬ 
quacy  of  Air  Force  FY02  DD350  data  for  spend  analyses  and  our 
findings  about  the  sufficiency  of  the  current  list  of  PSCs  to  describe 
Air  Force  purchases. 

Although  our  analyses  highlight  serious  problems  with  the 
DD350  data,  we  believe  that  the  Air  Force  and  the  Department  of 
Defense  (DoD)  could  take  steps  over  the  short  term  and  long  term  to 
improve  the  usefulness  of  these  data  for  conducting  spend  analyses. 

Over  the  short  term,  it  would  be  helpful  to  communicate  to  the 
entire  contracting  workforce  that  these  data  now  have  an  additional, 
important  purpose — to  aid  in  performing  analyses  to  support  imple¬ 
mentation  of  new  purchasing  and  supply  management  strategies. 
Our  hope  is  that  this  message  would  encourage  greater  precision  in 
describing  the  types  of  goods  and  services  purchased.  In  addition,  the 
Air  Force  could  collect  more  detailed  data  to  supplement  DD350 
data  on  contracts  that  fall  within  certain  “problem”  PSC  areas  that  we 
identify  in  this  report.  With  enough  additional  data,  the  Air  Force 
may  be  able  to  use  statistical  analyses  to  develop  guidelines  for 

Table  S.l 

Summary  of  Findings  Regarding  Accuracy  and  Adequacy  of  Air  Force  FY02 


DD350  Data 

Issue 

Findings 

Accuracy  of  the  PSC  coding  in  the 
DD350  forms  (see  pages  29-38) 

The  PSC  for  50  percent  of  contract  actions 
(39  percent  of  contract  dollars)  in  the  DD350 
database  is  coded  inaccurately. 

Services  are  undercounted. 

Adequacy  of  using  a  single  PSC  to 
describe  the  contract  action 
(see  pages  38-44) 

Eleven  percent  of  contract  actions  (27  percent  of 
contract  dollars)  include  the  purchase  of  more 
than  one  category  of  goods  and/or  services. 

Use  of  more  than  one  PSC  would  allow  for  better 
description  of  5  to  1 1  percent  of  contract 
dollars  in  the  DD350  database. 

How  well  the  available  PSCs 
describe  Air  Force  purchases 
(see  pages  22-26,  44) 

Several  important  categories  of  activities  are  not 
fully  captured  in  the  current  PSC  codes. 

New  PSCs  could  be  used  to  better  identify  how 
at  least  5  to  6  percent  of  contract  dollars  are 
spent. 
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adjusting  the  DD350  data  to  more  accurately  reflect  the  nature  of  the 
underlying  purchases  (see  page  52). 

Over  the  long  term,  the  Air  Force  may  be  able  to  work  with 
other  branches  of  the  military  and  federal  agencies  to  refine  the  list  of 
PSCs  to  include  codes  that  better  describe  Air  Force  purchases,  par¬ 
ticularly  for  warranties  on  goods  or  services,  special  studies  of  Air 
Force  operations  or  systems,  and  professional  services.  The  Air  Force 
may  also  want  to  consider  recommending  to  the  DoD  that  the 
DD350  form  be  modified  to  allow  contracting  officers  to  use  more 
than  one  PSC  to  describe  the  goods  and  services  purchased  through 
contract  actions  (see  page  52). 

Based  on  our  analyses,  modifying  the  DD350  form  to  allow  for 
additional  PSCs  and  their  corresponding  dollar  amounts  would  en¬ 
able  the  Air  Force  to  more  accurately  describe  how  5  to  1 1  percent  of 
DD350  contract  dollars  are  spent,  which  translates  to  $2  billion  to  $5 
billion  worth  of  expenditures  for  FY02.  Fiowever,  before  recom¬ 
mending  that  the  DD350  form  be  modified  to  allow  for  multiple 
PSCs,  the  Air  Force  should  evaluate  the  costs  (e.g.,  the  cost  of  making 
software  changes,  writing  new  manuals,  conducting  training  on  new 
systems,  entering  additional  data  in  the  form,  and  building  a  consen¬ 
sus  for  change  within  DoD)  versus  the  benefits  (i.e.,  the  value  added 
from  improved  purchasing  strategies  based  on  more  accurate  informa¬ 
tion  on  the  full  range  of  purchases)  of  such  a  change  (see  pages 

52-53). 

Finally,  the  Air  Force  may  benefit  from  providing  training  in 
PSC  coding  to  contracting  officers,  particularly  those  who  work  with 
technically  complex  contracts  or  contracts  that  include  many  differ¬ 
ent  types  of  purchases  (see  page  53). 
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CHAPTER  ONE 


Introduction 


A  robust  and  growing  set  of  literature  documents  the  evolution  of 
purchasing  and  supply  management  practices  within  the  private  sec¬ 
tor.1  Firms  are  analyzing  their  corporate-wide  expenditures  and  their 
supply  base  to  (1)  identify  cost-saving  opportunities,  (2)  develop  im¬ 
proved  purchasing  strategies,  and  (3)  better  manage  the  supply  base 
for  important  classes  of  goods  and  services.  The  goals  of  this  sort  of 
analysis  are  to  increase  performance  of  suppliers  (e.g.,  to  improve 
quality  and  to  increase  the  frequency  of  on-time  delivery)  and  reduce 
costs  in  order  to  extract  greater  value  from  the  goods  and  services  that 
firms  purchase. 

For  several  years,  the  Air  Force  has  been  working  to  incorporate 
well-regarded  commercial  purchasing  and  supply  management  prac¬ 
tices  for  important  categories  of  its  equipment  and  supplies.  RAND 
Project  AIR  FORCE  helped  initiate  and  is  now  supporting  these  eh 
forts,  particularly  the  Air  Force  Materiel  Command’s  pilot  projects 
for  improving  acquisition  of  spares  and  equipment  within  its  Air  Lo¬ 
gistics  Centers.2 

Supplies  and  other  goods  are  not  the  only  targets  of  opportunity 
for  the  Department  of  Defense  (DoD)  to  implement  improved  pur¬ 
chasing  and  supply  management  practices.  According  to  previous 


1  See  Moore  et  al.  (2002)  and  Ausink  et  al.  (2004)  for  discussions  of  this  literature. 

2  Leftwich  et  al.  (2004)  and  U.S.  General  Accounting  Office  (GAO;  now  the  Government 
Accountability  Office)  (2003a). 
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studies,3  DoD  spends  more  on  services  than  on  supplies  and  equip¬ 
ment.  Its  service  expenditures  have  grown  significantly  since  the  early 
1990s;  those  earlier  studies  show  that  in  fiscal  year  (FY)  2002,  the 
DoD  spent  about  $93  billion  on  services,  which  represents  an  in¬ 
crease  of  18  percent  from  FY01.  These  purchases  represent  a  broad 
range  of  service  activities  with  varying  degrees  of  complexity,  includ¬ 
ing  professional,  administrative  and  management  support;  repair  and 
maintenance  of  equipment  and  facilities;  information  technology 
support;  medical  services;  transportation  and  travel;  special  technical 
studies;  and  support  for  system  development  and  sustainment  activi¬ 
ties.  As  service  expenditures  have  increased  and  surpassed  other  types 
of  purchases,  Congress  has  placed  greater  emphasis  on  lowering  the 
cost  and  improving  the  performance  outcomes  of  purchased  services 
within  the  DoD.4 


Using  Spend  Analyses  to  Improve  Purchasing  and  Supply 
Management  Practices 

The  first  step  toward  implementing  improvements  in  purchasing  and 
supply  management  is  to  conduct  what  are  called  spend  analyses  to 
determine  the  nature  of  an  organization’s  expenditures  on  goods  and 
services  and  its  supply  base  (i.e.,  the  firms  that  sell  goods  and  services 
to  the  organization).5  Initial  aggregate-level  analyses  are  used  to  help 
build  the  case  for  changing  purchasing  and  supply  management  prac¬ 
tices  to  better  support  corporate  goals  and  to  highlight  potential  tar¬ 
gets  of  opportunity  for  further,  more-detailed  analyses.  These  more- 


3  GAO  (2003b,  2003a,  2002,  and  2001);  The  Next  Steps  in  Services  Acquisition  Reform  .  .  . 
(2001). 

4  The  National  Defense  Authorization  Act  (NDAA)  for  Fiscal  Year  2002  mandated  imple¬ 
mentation  of  improved  management  practices  for  DoD  service  acquisitions,  and  it  estab¬ 
lished  savings  goals  to  be  met  over  ten  years.  To  motivate  the  use  of  new  management  prac¬ 
tices,  the  FY03  version  of  the  NDAA  reduced  the  FY03  authorization  for  purchased  services 
in  DoD  by  $600  million,  with  the  Air  Force’s  share  of  the  reduction  being  $183  million.  See 
National  Defense  Authorization  Act  for  Fiscal  Year  2003  (2002). 

5  Moore  et  al.  (2002  and  2004);  GAO  (2003a). 
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detailed  analyses  are  then  used  to  develop  improved  purchasing 
strategies  that  are  tailored  to  the  characteristics  of  the  products  being 
purchased  (including  the  products’  supply  chains  and  characteristics 
of  the  products’  industries),  the  organization’s  demand  for  those 
products  (e.g.,  the  amount  purchased,  the  range  of  internal  custom¬ 
ers,  the  variation  in  product  specifications  needed,  the  products’ 
importance  to  core  activities),  and  the  organization’s  corporate  objec¬ 
tives  (e.g.,  making  low-cost  products  versus  maintaining  a  techno¬ 
logical  advantage) .  As  spend  analyses  are  repeated  over  time,  they  can 
also  help  an  organization  determine  whether  the  introduction  of  new 
practices  is  achieving  the  desired  results.  An  analysis  conducted  before 
such  changes  are  introduced  can  serve  as  a  baseline  against  which  to 
compare  analyses  conducted  after  reforms  are  made. 

For  example,  “rationalizing”  the  number  of  suppliers  is  an  im¬ 
portant  part  of  many  firms’  long-term  purchasing  strategy.6  Ration¬ 
alizing  involves  determining  the  “right”  number  of  suppliers  for  a 
company  and  could  include  decreasing  or  increasing  the  number  of 
suppliers  that  provide  a  given  good  or  service.  A  company  with  too 
many  suppliers  for  a  given  good  or  service  may  not  have  sufficient 
leverage  over  any  individual  supplier  to  reduce  costs  or  increase  per¬ 
formance.7  On  the  other  hand,  a  company  with  too  few  suppliers  of  a 
particular  good  or  service  could  be  at  risk  of  poor  supplier  perform¬ 
ance  if  suppliers  do  not  feel  any  competitive  pressure  to  control  costs, 
improve  their  products,  or  deliver  on  time.  The  “right”  number  of 
suppliers  will  depend  on  the  importance  of  the  good  or  service  to  the 
firm  and  the  risks  that  are  presented  if  the  supply  of  the  item  is  inter¬ 
rupted.  A  spend  analysis  to  determine  how  many  suppliers  are  being 
used  and  how  much  they  are  being  paid  is  an  important  step  in  ra¬ 
tionalizing  the  supply  base. 

The  Air  Force  Deputy  Assistant  Secretary  for  Contracting 
(SAF/AQC)  asked  RAND  Project  AIR  FORCE  to  conduct  a  spend 


6MacLean  (2002). 

7  A  large  supply  base  also  makes  it  more  difficult  to  form  strategic  supplier  relationships  and 
undertake  supplier-development  activities  (e.g.,  improve  supplier  quality  assurance  proce¬ 
dures  or  implement  cost-saving  production  techniques). 
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analysis  of  the  Air  Force’s  expenditures  on  purchased  services  to  sup¬ 
port  its  efforts  in  implementing  proven  commercial  purchasing  and 
supply  management  practices.  To  determine  whether  an  accurate 
spend  analysis  could  be  conducted,  we  needed  to  learn  more  about 
the  available  data  on  Air  Force  contracts.  As  discussed  in  the  next  sec¬ 
tion  of  this  chapter,  questions  about  the  reliability  of  Air  Force  con¬ 
tract  data  motivated  the  detailed  analysis  presented  in  this  report.  Af¬ 
ter  we  began  our  analysis  of  the  contract  database,  the  Office  of  the 
Secretary  of  Defense  began  its  own  DoD-wide  spend  analyses.8 


The  DD350  Form  and  Spend  Analyses 

The  primary  source  of  data  for  Air  Force  and  DoD  contract  expendi¬ 
tures  is  the  Individual  Contracting  Action  Report  form,  also  known 
as  the  DD350  (see  Appendix  A  for  the  data  fields  in  a  DD350).  A 
DD350  form  is  completed  for  each  DoD  contract  transaction9  in¬ 
volving  purchases  in  excess  of  $25,000  and  contains  descriptive  in¬ 
formation  about  each  purchase.10  DD350  forms  are  filled  out  by  Air 
Force  contracting  officers,  and  all  Air  Force  data  on  contract  actions 
are  submitted  electronically  to  a  central  database  through  a  reporting 
system  called  “J001.” 

The  DD350  form  and  the  data  system  into  which  the  form  is 
fed  were  not  designed  for  the  purpose  of  supporting  spend  analyses. 
Air  Force  documents  state  that  the  Executive  Branch,  Congress,  the 
General  Accounting  Office  (now  called  the  Government  Account- 


8  GAO  (2003b). 

9  Contract  transactions  are  hereafter  referred  to  in  this  report  as  contract  actions.  A  single 
contract  between  the  Air  Force  and  a  supplier  can  have  multiple  contract  actions  associated 
with  it.  Such  actions  may  represent  incremental  obligations  of  funds,  deobligations,  orders 
against  indefinite  delivery  and  indefinite  quantity  (IDIQ)  contracts,  and  other  contract 
modifications. 

10  Use  of  the  DD350  form  is  governed  by  Defense  Federal  Acquisition  Regulation  Supple¬ 
ment  (DFARS)  204.670-2.  Instructions  for  filling  out  the  form  are  found  in  DFARS 
253.204-70.  Purchases  for  less  than  $25,000  are  typically  recorded  in  a  DD1057  form,  al¬ 
though  they  may  be  recorded  in  the  DD350  instead.  In  some  special  cases,  actions  greater 
than  $25,000  are  recorded  in  the  DD1057. 
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ability  Office),  and  the  Small  Business  Administration  use  the  data  to 
study  DoD  purchases  from  small  or  small  and  disadvantaged  busi¬ 
nesses,  and  to  determine  other  information  about  competition  for 
contracts.11  As  such,  the  DD350  data  suffer  from  several  limitations 
when  viewed  from  a  spend  analysis  perspective: 

•  The  DD350  form  allows  only  one  principal  product/service 
code  (PSC)  to  be  used  to  characterize  each  contract  action.  A 
contract  action  may  contain  multiple  services  and/or  goods; 
therefore,  the  DD350  data  may  overstate  or  understate  the  types 
of  goods  or  services  actually  purchased. 

•  The  entire  federal  government  uses  the  same  set  of  PSCs  to  de¬ 
scribe  its  purchases,  so  finding  a  code  that  precisely  matches  a 
good  or  service  that  is  purchased  solely  by  the  military  may  be 
difficult. 

•  Interviews  we  conducted  with  Air  Force  personnel  during  the 
course  of  this  study  indicated  that  contracting  officers  receive  lit¬ 
tle  guidance  on  selecting  an  appropriate  PSC;  therefore,  they  use 
their  own  discretion  when  filling  in  the  DD350  data  fields.  Be¬ 
cause  filling  out  the  DD350  form  is  considered  to  be  the  final 
formality  in  completing  a  contract  action,  busy  contracting  offi¬ 
cers  may  not  spend  a  lot  of  time  on  this  task. 

•  Interviews  also  suggested  that  the  DD350  system  is  difficult  to 
maneuver  and  sometimes  rejects  inputs  that  contracting  officers 
feel  are  most  appropriate  due  to  internal  consistency  checks  that 
are  not  transparent  to  users.  With  few  formal  audits  of  the  accu¬ 
racy  of  the  DD350  data,  it  is  reasonable  to  believe  that  informa¬ 
tion  on  a  transaction  recorded  in  the  DD350  database  may 
sometimes  differ  from  the  actual  facts  of  the  transaction.12 


11  U.S.  Air  Force  Instruction  64-105  (2000). 

12We  did  find  documentation  on  one  audit  of  DD350  data,  which  was  conducted  by  the 
DoD  Inspector  General — Waivers  of  Requirement  for  Contractors  to  Provide  Cost  or  Pric¬ 
ing  Data,  (2001).  However,  the  report  analyzed  only  one  block  of  the  DD350  form,  which 
was  related  to  the  Truth  in  Negotiation  Act  (TINA). 
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Because  of  concerns  that  we,  and  other  researchers,  have  had 
about  potential  inadequacies  and  inaccuracies  in  these  data,13we  for¬ 
mally  assessed  the  degree  of  confidence  one  should  have  in  key  data 
fields  that  describe  Air  Force  purchases.  Although  our  original  focus 
was  on  expenditures  for  services,  concerns  about  the  accuracy  of  the 
data  (in  particular,  the  possibility  that  services  are  coded  as  goods  or 
embedded  in  contracts  for  purchases  of  goods,  for  example)  led  us  to 
broaden  our  analyses  and  examine  data  for  both  goods  and  services 
contracts.  This  report  describes  our  research  approach  and  methodol¬ 
ogy,  as  well  as  our  findings  on  the  adequacy  and  accuracy  of  Air  Force 
DD350  data  for  conducting  spend  analyses. 


Research  Approach 

DD350  forms  correspond  to  contract  actions  rather  than  complete 
contracts;  thus,  our  unit  of  analysis  is  the  contract  action.  The  num¬ 
ber  of  Air  Force  contract  actions  recorded  in  the  DD350  database  is 
very  large — more  than  65,000  actions  were  recorded  for  FY02.  We 
selected  a  moderately  sized  sample  of  306  Air  Force  contract  actions 
from  FY02  and  interviewed  the  contracting  officers  who  (according 
to  the  data)  had  submitted  the  information.  We  took  this  approach 
in  order  to  learn  more  about  the  nature  of  each  contract  action,  in¬ 
cluding  what  was  purchased,  the  dollar  amount,  and  other  relevant 
contract  information.  After  analyzing  the  results  from  the  interviews, 
we  used  a  statistical  approach  to  reweight  our  findings  and  extrapo¬ 
late  the  results  to  the  entire  Air  Force  FY02  DD350  population.  (Our 
methodology  is  described  in  detail  in  Chapter  Two.)  Our  goals  were 
to  determine  the  adequacy  of  the  DD350  data  (i.e. ,  how  completely 
one  PSC  described  what  was  purchased  by  a  contract  action)  and  the 
accuracy  of  the  data  (i.e.,  how  closely  what  was  recorded  in  the  data 


13  A  1999  study  by  Litton-TASC  found  that  users  were  concerned  about  the  integrity  of  the 
DD350  database.  Users  also  found  it  difficult  to  correct  errors  made  in  the  system  (Litton- 
TASC,  1999).  See  also  Moore  et  al.  (2004)  and  GAO  (2003a). 
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matched  what  we  learned  in  interviewing  the  contracting  officer  re¬ 
sponsible  for  the  action). 


Primary  Findings 

Our  detailed  findings,  including  a  description  of  the  potential  causes 
of  problems  with  the  Air  Force  DD350  data,  and  suggestions  for  im¬ 
proving  data  collection  are  discussed  in  the  following  chapters.  Here, 
we  highlight  our  primary  findings,  extrapolated  from  our  survey  data 
for  FY02. 

•  The  PSC  for  50  percent  of  contract  actions  (representing  39 
percent  of  the  total  dollar  value  of  the  contract  actions)  is  coded 
inaccurately;  we  estimate  that  services  account  for  66  percent  of 
Air  Force  purchases,  rather  than  5 1  percent  as  indicated  by  the 
DD350  data. 

•  More  than  one  distinct  good  and/or  service  was  purchased  in  1 1 
percent  of  the  contract  actions  (representing  27  percent  of  the 
dollar  value  of  the  contract  actions  in  the  DD350  database);  5  to 
1 1  percent  of  the  dollar  value  of  all  actions  (roughly  $2  billion 
to  $5  billion)  were  associated  with  activities  not  described  by  the 
primary  PSC  (i.e.,  secondary  activities). 

•  New  PSCs  could  be  introduced  to  better  describe  several  types 
of  common  purchases;  these  purchases  represent  at  least  5  to  6 
percent  of  overall  contract  dollars. 


Organization  of  This  Report 

Chapter  Two  describes  our  research  methodology  in  detail.  It  in¬ 
cludes  a  description  of  the  DD350  form  and  our  sampling  method- 
ology,  problem  areas  uncovered  during  our  survey,  and  a  general  dis¬ 
cussion  of  the  statistical  methods  used  to  extrapolate  the  survey 
results.  Chapter  Three  presents  the  results  of  our  analysis  of  the  accu¬ 
racy  and  adequacy  of  the  Air  Force  FY02  DD350  data  for  conducting 
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spend  analyses.  Chapter  Four  summarizes  our  findings  and  includes 
recommendations  for  steps  the  Air  Force  might  take  in  the  short  and 
long  term  to  improve  the  usefulness  of  DD350  data  for  spend 
analyses. 

Appendix  A  contains  data  fields  from  the  DD350  form.  Appen¬ 
dix  B  lists  the  PSC  categories  used  in  our  analyses.  Appendix  C  con¬ 
tains  the  survey  questionnaire  we  used  to  gather  information  from  Air 
Force  contracting  officers.  Appendix  D  describes  the  contract  actions 
included  in  the  survey.  Appendix  E  details  the  statistical  approach  we 
used  to  extrapolate  the  findings  from  our  survey  to  all  DD350  con¬ 
tract  actions. 


CHAPTER  TWO 


Research  Methodology 


Our  analysis  of  the  accuracy  and  adequacy  of  the  Air  Force  FY02 
DD350  database  rests  on  an  examination  of  a  sample  of  contract  ac¬ 
tions  from  the  DD350.  In  this  chapter,  we  first  describe  how  we  se¬ 
lected  this  sample,  the  survey  methods  we  used  to  collect  additional 
information  from  the  contracting  officers  responsible  for  supplying 
the  data  on  the  contract  actions  in  our  sample,  and  the  survey  re¬ 
sponse  rate.  We  then  describe  some  of  the  issues  that  arose  in  coding 
and  interpreting  the  information  provided  by  the  contracting  officers. 
We  conclude  by  briefly  discussing  the  statistical  methods  we  used  to 
extrapolate  the  findings  from  the  sample  to  the  Air  Force  FY02  con¬ 
tract  actions  in  the  DD350  database  as  a  whole  (which  are  discussed 
in  more  depth  in  Appendix  E). 


Overview  of  DD350  Database 

The  DD350  database  is  the  primary  source  of  information  on  the 
contracts  the  Air  Force  enters  into  when  it  purchases  goods  and 
services.  A  single  contract  between  the  Air  Force  and  a  supplier  can 
have  multiple  contract  actions  associated  with  it.  Actions  may  repre¬ 
sent  obligations  under  new  contracts,  additional  funds  to  existing 
contracts,  deobligations  of  funds,  orders  under  indefinite  delivery  and 
indefinite  quantity  (IDIQ)  contracts,  and  other  contract  modifica¬ 
tions.  The  DD350  database  provides  information  on  virtually  every 
DoD  contract  action  involving  a  purchase  worth  more  than  $25,000. 
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The  contract  actions  in  the  DD350  database  cover  a  substantial 
percentage  of  total  Air  Force  purchases  of  goods  and  services  and 
nearly  all  direct  purchases  from  commercial  firms.  Moore  et  al. 
(2004)  put  total  Air  Force  purchases  for  FY02  at  $69  billion.  Ap¬ 
proximately  28  percent  of  those  purchases  were  made  from  other 
government  agencies,  leaving  roughly  $50  billion  in  direct  purchases 
from  commercial  firms.  Moore  et  al.  estimate  that  96  percent  of  the 
direct  commercial  purchases  are  captured  on  DD350  forms.1 

The  DD350  database  provides  a  substantial  amount  of  informa¬ 
tion  on  each  contract  action,  including  the  following: 

•  dollar  amount  of  the  contract  action 

•  limited  description  of  what  was  purchased 

•  purchasing  organization 

•  winning  contractor 

•  date  of  funding  or  period  of  performance 

•  type  of  contract  pricing  (e.g.,  firm-fixed-price  or  cost-plus- 
incentive-fee) 

•  type  of  competition  (e.g.,  full  and  open  or  set-aside) 

•  type  of  contractor  (e.g.,  small  disadvantaged  business,  woman- 
owned  business,  business  owner  of  a  particular  ethnic  group) 

•  contracting  officer  and  his  or  her  contact  information. 

The  DD350  form  describes  what  was  purchased  using  a  single 
PSC,  which  is  a  four-character  hierarchical  code.  The  general  cate¬ 
gory  of  expenditures  is  described  by  the  PSC’s  first  character  (which 
can  be  a  letter  or  digit)  and  the  remaining  characters  provide  more 
details  on  what  was  purchased  within  the  general  category.  According 
to  the  PSC  documentation,  PSCs  that  begin  with  a  letter  correspond 
to  categories  of  services,2  and  PSCs  that  begin  with  a  number  corre- 


1  Contract  actions  for  less  than  $25,000  (captured  in  the  DD1057  system)  account  for  less 
than  1  percent  of  direct  commercial  purchases,  and  purchases  using  government  purchase 
cards  (credit  cards)  account  for  the  remaining  3  percent. 

2  There  is  vigorous  debate  within  the  Air  Force  Materiel  Command  about  what  constitutes  a 
service.  For  example,  research  and  development  (the  PSC  for  which  begins  with  the  letter  A) 
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spond  to  categories  of  goods  (U.S.  General  Services  Administration, 
1998). 3  Table  2.1  defines  the  first  character  of  the  PSCs  that  we  used 
in  our  analysis.  More  details  on  PSCs  are  provided  in  Appendix  B. 

Tables  2.2  and  2.3  provide  an  overview  of  the  number  and 
dollar  value  of  Air  Force  contract  actions  in  the  DD350  database  for 
FY02.  Approximately  65,500  Air  Force  contract  actions,  having  a 

Table  2.1 

PSC  Categories  Used  in  the  Analysis 


First  Character  of 
PSC 

Description  of  Goods  or  Services  in  the  Category 

Goods 

1 

Weapons,  ammunition,  aircraft,  space  vehicles 

2 

Engines,  turbines,  ground  vehicles 

3  to  9 

Other  goods 

Services 

A 

Research  and  development  (R&D) 

B 

Special  studies  and  analyses  (not  R&D) 

D 

Automatic  data  processing  and  telecommunication  services 

J 

Maintenance,  repair,  and  rebuilding  of  equipment 

K 

Modification  of  equipment 

L 

Technical  representative  services 

M 

Operation  of  government-owned  facility 

N 

Installation  of  equipment 

R 

Professional,  administrative,  and  management  support 
services 

S 

Utilities  and  housekeeping  services 

U 

Education  and  training 

Other  letters 

Other  services 

is  not  considered  by  some  Air  Force  product  center  personnel  to  be  a  service,  even  though  it 
is  considered  a  service  in  the  PSC  classification  system.  Similarly,  component-repair  activities 
(the  PSC  for  which  begins  with  the  letter  J)  are  not  considered  to  be  services  by  some  within 
the  Air  Force  Air  Logistics  Centers.  See  Ausink  et  al.  (2002)  for  additional  discussion  of  these 
issues.  We  accept  the  PSC  definitions  of  goods  and  services.  Thus,  we  consider  all  activities 
that  begin  with  a  letter  in  the  PSC  system  to  be  services  and  all  those  that  begin  with  a  num¬ 
ber  to  be  goods. 

3  The  PSC  field  for  the  contract  actions  in  the  completed  DD350  forms  we  examined  for 
this  study  was  never  blank,  and  PSC  9999  (for  items  that  could  not  be  classified  using  any 
other  PSC)  appeared  on  only  a  few  actions. 


Table  2.2 

Distribution  of  Air  Force  Contract  Actions  in  the  DD350  Database  for  FY02,  by  First  Character  of  the  PSC  and  Dollar 
Value  of  the  Contract  Action 


Size  of  Contract  Action 

Total 

Number  of 
Contract 
Actions 

Share  of  All 

First  Character  of  PSC  and 
Type  of  Contract  Action 

S0-S100K 

S100K- 

S500K 

S500K- 

$1M 

$1M- 

$10M 

$10M- 

$50M 

$50M- 

$100M 

>  $100M 

Contract 
Actions  in 
DD350 

Database  (%) 

Goods 

1  Aircraft  and 

1,366 

1,245 

382 

803 

140 

17 

20 

3,973 

6 

components 

2  Engines  and 

659 

559 

225 

354 

37 

2 

1 

1,837 

3 

components 

Other  goods 

10,191 

3,673 

672 

766 

48 

5 

0 

15,355 

23 

Total  goods 

12,216 

5,477 

1,279 

1,923 

225 

24 

21 

21,165 

32 

Services 

A  R&D 

3,443 

3,299 

699 

787 

139 

15 

10 

8,392 

13 

B  Special  studies,  not 

334 

297 

94 

97 

5 

1 

0 

828 

1 

R&D 

D  Automatic  data 

1,186 

818 

221 

175 

6 

0 

0 

2,406 

4 

processing  and 
telecom  services 

J  Maintenance  and 

2,146 

1,505 

352 

407 

19 

0 

0 

4,429 

7 

repair 
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Table  2.2 — Continued 


Size  of  Contract  Action 

Total 

Number  of 
Contract 
Actions 

Share  of  All 

First  Character  of  PSC  and 
Type  of  Contract  Action 

S0-S100K 

$100K- 

S500K 

S500K- 

$1M 

Si  M- 
$10M 

$10M- 

$50M 

$50M- 

$100M 

>  $100M 

Contract 
Actions  in 
DD350 

Database  (%) 

Services  (continued) 

K  Modification  of 

128 

185 

91 

0 

0 

464 

equipment 

54 

6 

1 

L  Technical 
representative 
services 

125 

157 

41 

48 

12 

2 

2 

387 

1 

M  Operation  of 

government-owned 

facility 

238 

273 

55 

87 

9 

2 

0 

664 

1 

R  Support  services 

2,338 

2,289 

588 

686 

32 

3 

0 

5,936 

9 

S  Utilities  and 
housekeeping 

1,579 

1,009 

253 

248 

17 

0 

0 

3,106 

5 

Other  Services 

11,598 

4,546 

869 

670 

34 

3 

1 

17,721 

27 

Total  Services 

23,115 

14,378 

3,226 

3,296 

279 

26 

13 

44,333 

68 

Total  Goods  and  Services 

35,331 

19,855 

4,505 

5,219 

504 

50 

34 

65,498 

100 

Share  of  All  Contract 
Actions  in  DD350 
Database  (%) 

54 

30 

7 

8 

1 

0.1 

0.1 

100 

NOTES:  Some  rows  and  columns  do  not  sum  exactly  due  to  rounding.  K  =  1,000;  M  =  1,000,000. 
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Table  2.3 

Value  of  Air  Force  Contract  Actions  in  the  DD350  Database  for  FY02  ($millions),  by  First  Character  of  the  PSC  and 
Dollar  Value  of  the  Contract  Action 


Size  of  Contract  Action  Share  of 


Value  of  All 

Total  Value  Contract 


First  Character  of  PSC  and  Type 
of  Contract  Action 

$0-$100K 

$100K- 

S500K 

$500K- 

$1M 

$1M- 

$10M 

$10M- 

$50M 

S50M- 

S100M 

>  $100M 

of  Contract 
Actions 

Actions  in 
DD350  (%) 

Goods 

1  Aircraft  and  components 

71 

299 

279 

2,795 

2,756 

1,163 

8,928 

16,291 

33 

2  Engines  and  components 

33 

137 

164 

1,013 

686 

150 

308 

2,491 

5 

Other  Goods 

479 

809 

474 

2,089 

845 

320 

0 

5,015 

10 

Total  Goods 

583 

1,245 

916 

5,896 

4,287 

1,633 

9,237 

23,797 

49 

Services 

A  R&D 

208 

849 

508 

2,376 

2,855 

947 

2,060 

9,803 

20 

B  Special  studies,  not  R&D 

17 

71 

66 

204 

98 

66 

0 

522 

1 

D  Automatic  data  processing 

55 

193 

155 

418 

124 

0 

0 

945 

2 

and  telecom  services 

J  Maintenance  and  repair 

109 

366 

247 

963 

363 

0 

0 

2,049 

4 

K  Modification  of  equipment 

8 

51 

39 

239 

116 

0 

0 

453 

1 

L  Technical  representative 

6 

38 

29 

150 

235 

150 

389 

997 

2 

services 
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Table  2.3 — Continued 


Size  of  Contract  Action  Share  of 


Value  of  All 

Total  Value  Contract 


First  Character  of  PSC  and  Type 
of  Contract  Action 

$0-$100K 

$100K- 

$500K 

$500K- 

$1M 

$1M- 

S10M 

$10M- 

$50M 

S50M- 

S100M 

>  $100M 

of  Contract 
Actions 

Actions  in 
DD350 (%) 

Services  (continued) 

M  Operation  of  government- 

13 

66 

40 

275 

202 

114 

0 

710 

1 

owned  facility 

R  Support  services 

122 

552 

425 

1,870 

550 

254 

0 

3,773 

8 

S  Utilities  and  housekeeping 

70 

238 

177 

521 

280 

0 

0 

1,287 

3 

Other  Services 

390 

1,089 

617 

1,648 

585 

173 

115 

4,617 

9 

Total  Services 

999 

3,515 

2,303 

8,662 

5,408 

1,703 

2,563 

25,154 

51 

Total  Goods  and  Services 

1,582 

4,759 

3,220 

14,559 

9,695 

3,336 

11,800 

48,951 

100 

Share  of  Value  of  All 

3 

10 

7 

30 

20 

7 

24 

100 

Contract  Actions  in  DD350 

Database  (%) 

NOTES:  Some  rows  and  columns  do  not  sum 

exactly  due  to  rounding.  K  = 

1,000;  M  = 

1,000,000. 
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total  purchase  value  of  nearly  $49  billion,  are  in  the  DD350  database 
for  FY02.  The  tables  show  the  following: 

•  According  to  the  PSCs,  there  are  roughly  twice  as  many  contract 
actions  for  services  as  there  are  for  goods,  although  goods  and 
services  each  account  for  roughly  one-half  the  total  contract- 
action  dollars. 

•  A  high  percentage  (91  percent)  of  the  contract  actions  are  for 
less  than  $1  million,  but  the  dollar  value  of  purchases  is  concen¬ 
trated  in  contract  actions  for  more  than  $1  million.  Contract  ac¬ 
tions  exceeding  $1  million  account  for  9  percent  of  the  contract 
actions  but  80  percent  of  the  contract-action  dollars. 

•  Contract  actions  for  services  tend  to  be  smaller  than  those  for 
goods,  with  the  result  that  more  than  60  percent  of  contract  ac¬ 
tions  are  for  services  costing  less  than  $1  million.  These  contract 
actions  account  for  less  than  15  percent  of  total  contract  dollars. 

Figures  2.1  and  2.2  graphically  illustrate  the  distribution  of  con¬ 
tract  actions  and  contract-action  dollars  by  PSC  category. 

Figure  2.3  plots  the  cumulative  percentage  of  contract  dollars 
against  the  cumulative  percentage  of  contract  actions.  It  clearly  illus¬ 
trates  the  concentration  of  purchase  dollars  within  relatively  few  con¬ 
tract  actions.  For  example,  it  shows  that  contract  actions  for  $10  mil¬ 
lion  or  more  account  for  roughly  1  percent  of  all  contract  actions  but 
approximately  50  percent  of  all  contract  dollars. 


Sampling  Methodology  and  Survey 

We  collected  supplemental  information  on  a  sample  of  Air  Force 
FY02  DD350  contract  actions  through  a  survey  of  Air  Force  con¬ 
tracting  officers  listed  in  the  DD350  database  as  being  responsible  for 
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Figure  2.1 

PSC  Categories  as  a  Percentage  of  Total  Air  Force  FY02  DD350  Contract 
Actions 


those  actions.  Our  approach  to  sample  selection  balanced  limited 
project  resources  with  the  need  for  a  sample  that  would  provide  as 
much  information  about  the  adequacy  and  accuracy  of  the  DD350 
data  as  possible.  To  this  end,  we  weighted  the  sample  toward  contract 
actions  with  larger  dollar  values  because  they  account  for  a  dispropor¬ 
tionately  large  share  of  overall  contract  expenditures,  and  because  we 
suspected  that  these  actions  might  contain  multiple  distinct  activities. 
We  conducted  telephone  interviews  rather  than  use  a  written  survey 
so  that  we  could  ask  questions  to  more  fully  explore  the  nature  of  the 
contract  actions.  To  reduce  the  cost  of  the  telephone  interviews,  we 
constructed  the  sample  so  that  we  could  collect  information  on  more 
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Figure  2.2 

PSC  Categories  as  a  Percentage  of  Total  Air  Force  FY02  DD350  Contract 
Dollars 
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components 
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than  one  contract  action  from  each  contracting  officer  we  inter¬ 
viewed.  In  this  section,  we  outline  our  process  for  selecting  the  sam¬ 
ple  and  describe  the  survey. 

Budget  and  time  constraints  limited  our  sample  to  approxi¬ 
mately  300  Air  Force  contract  actions  from  the  FY02  DD350  data¬ 
base.  As  discussed  earlier,  roughly  one-half  of  the  expenditures  were 
for  contract  actions  that  the  PSCs  indicated  were  services.  This  num¬ 
ber  might  suggest  that  splitting  the  sample  roughly  evenly  between 
goods  and  services  would  be  appropriate.  Flowever,  there  were  more 
than  twice  as  many  service  actions  in  the  DD350  as  there  were  goods 
actions.  Based  on  these  considerations  and  our  sponsor’s  particular 
interest  in  services,  we  selected  approximately  225  service  contract 
actions  and  75  goods  contract  actions  for  the  final  sample. 
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Figure  2.3 

Concentration  of  Air  Force  FY02  DD350  Contract  Expenditures 


We  used  a  two-stage  process  to  select  our  final  sample.  The  ini¬ 
tial  sample  was  randomly  selected  with  sampling  weights  proportional 
to  the  dollar  value  of  the  contract  actions  (weights  were  calculated 
separately  for  goods  and  services).4  In  the  second  stage,  we  selected 
contract  actions  from  those  identified  in  the  first  stage  using  the  crite¬ 
ria  discussed  next. 

Roughly  2,500  contracting  officers  are  listed  in  the  Air  Force 
FY02  DD350  database.  This  means  that  contracting  officers  on  aver¬ 
age  handle  just  over  25  contracting  actions  per  year.  In  our  initial 
sample,  some  contracting  officers  were  associated  with  multiple  con- 


4  To  make  the  selections  using  weights  proportional  to  the  dollar  value,  we  multiplied  the 
value  of  each  contract  action  by  a  constant.  We  then  assigned  a  random  number  between 
zero  and  one  to  each  contract  action  and  selected  the  contract  action  if  the  random  number 
was  less  than  the  constant  times  the  dollar  amount  of  the  action.  The  constant  was  adjusted 
until  the  desired  number  of  contract  actions  was  obtained.  The  same  approach  was  used  for 
the  goods  contract  actions. 
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tract  actions,  and  others  were  associated  with  only  one.  As  stated  ear¬ 
lier,  to  reduce  the  cost  of  conducting  the  interviews,  we  excluded  con¬ 
tracting  officers  with  only  one  action.  To  ensure  that  we  had  suffi¬ 
cient  heterogeneity  in  our  sample,  we  randomly  selected  two  contract 
actions  from  those  contracting  officers  who  were  responsible  for  three 
or  more  of  the  contract  actions  in  the  initial  random  sample.  Because 
we  wanted  to  make  sure  that  the  largest  contract  actions  were  in¬ 
cluded  in  the  sample,  however,  we  made  some  exceptions  to  this  pro¬ 
cedure.  All  goods  contract  actions  for  more  than  $100  million  were 
selected  from  the  initial  random  sample,  even  if  it  meant  that  there 
was  a  contracting  officer  in  the  final  sample  with  only  one  contract 
action.  If  a  service  contract  action  was  greater  than  $50  million,  and 
the  associated  contracting  officer  had  only  one  contract  action,  the 
action  was  still  included  in  the  final  sample.  This  did  not  mean  that 
all  service  actions  greater  than  $50  million  in  the  initial  sample  were 
included  in  the  final  sample;  some  contracting  officers  in  the  initial 
sample  were  responsible  for  more  than  two  $50-million  service  ac¬ 
tions,  in  which  case  only  two  of  their  contract  actions  were  randomly 
selected. 

In  the  end,  231  services  contract  actions  and  75  goods  contract 
actions  were  selected.  The  231  actions  amounted  to  0.5  percent  of 
the  44,333  services  contract  actions  and  22  percent  of  the  expendi¬ 
tures  on  services  contract  actions  in  the  Air  Force  FY02  DD350 
database.  Although  the  sample  of  goods  contract  actions  amounts  to 
only  0.35  percent  of  the  overall  goods  actions,  it  covers  42  percent  of 
the  dollars  in  the  FY02  DD350  data. 

Supplemental  information  on  the  actions  in  our  final  sample 
was  collected  through  structured  telephone  interviews  with  the  con¬ 
tracting  officers  listed  for  each  action  in  the  DD350  database.  (See 
Appendix  C  for  the  interview  protocol).5  Telephone  interviews  typi¬ 
cally  lasted  approximately  ten  minutes  per  contract  action,  or  20  to 


5  Initial  contact  was  made  with  contracting  officers  by  letter  (via  FedEx),  e-mail,  or  tele¬ 
phone.  The  purpose  of  the  study  was  described  and  the  name  of  a  contact  at  the  organization 
sponsoring  the  study  (SAF/AQC)  was  provided.  When  requested,  a  list  of  interview  ques¬ 
tions  was  provided  to  help  the  contracting  officer  prepare  for  the  interview. 
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25  minutes  per  contracting  officer.  The  interviews  were  conducted 
during  February  and  March  2003. 

As  shown  in  Table  2.4,  the  response  rate  to  the  survey  was  high. 
Of  the  306  contract  actions  selected,  interviews  were  completed  with 
the  contracting  officers  responsible  for  243  of  them  (79  percent). 
Very  few  contracting  officers  refused  to  participate  in  the  study.  We 
were  unable  to  contact  some  of  them,  and  some  were  unable  to 
schedule  interviews  before  the  survey  ended.  The  characteristics  of  the 
contract  actions  for  which  interviews  were  completed  are  summarized 
in  Appendix  D.  Overall,  expenditures  on  the  180  services  contract 
actions  for  which  we  completed  interviews  totaled  $4.4  billion  (17 
percent  of  expenditures  for  all  Air  Force  FY02  service  contract  actions 
in  the  DD350  database).  Expenditures  on  the  63  goods  contract  ac¬ 
tions  for  which  we  completed  interviews  totaled  $9.6  billion  (40  per¬ 
cent  of  expenditures  for  all  Air  Force  FY02  goods  contract  actions  in 
the  DD350  database). 


Challenges  in  Assigning  PSCs  to  Contract  Actions 


After  interviewing  each  contracting  officer,  we  assigned  a  PSC  to  the 


contract  action  for  which  the 

Table  2.4 

Survey  Completion  Rate 

contracting 

officer  was 

responsible 

Number  of 
Service 
Actions 

Number  of 
Goods 
Actions 

Total 

Contract  actions  selected  from  DD350 
database 

231 

75 

306 

Contracting  officer  refused  to 
participate 

5 

2 

7 

Unable  to  contact  contracting  officer 
or  schedule  an  interview 

46 

10 

56 

Contract  actions  for  which  interview 
was  completed 

180 

63 

243 

Completion  rate 

78% 

84% 

79% 

22  An  Assessment  of  Air  Force  Data  on  Contract  Expenditures 


based  on  his  or  her  description  of  the  goods  and/or  services  purchased 
(these  PSCs  are  referred  to  in  this  report  as  “RAND-assigned”  PSCs). 

We  found  this  code-assignment  process  to  be  challenging  be¬ 
cause  of  limitations  in  the  available  PSCs  and  the  complexity  and 
range  of  activities  that  the  Air  Force  purchases.  In  some  cases,  it  ap¬ 
peared  that  no  existing  code  accurately  described  an  activity.  In  other 
cases,  several  different  codes  could  arguably  be  applied  to  the  same 
activity.  To  ensure  consistency  in  the  PSC  assignments,  four  RAND 
researchers  with  previous  Air  Force  contracting-related  research  expe¬ 
rience  conducted  the  interviews.  The  entire  research  team  partici¬ 
pated  in  an  initial  set  of  interviews  to  standardize  the  interview 
process  and  the  coding  of  questionnaire  responses — in  particular,  the 
assignment  of  PSCs  to  activities  as  they  were  described  by  contracting 
officers  during  the  interviews.  Subsequent  interviews  were  conducted 
by  individual  team  members,  but  weekly  teleconferences  were  con¬ 
ducted  to  discuss  progress  and  problems,  and  to  ensure  that  all  re¬ 
searchers  were  using  the  same  rules  to  assign  codes.  The  coding  was 
most  challenging  when  we  found  that  the  action  included  more  than 
one  service  or  good  or  a  combination  of  goods  and  services.  In  some 
cases,  follow-up  conversations  with  contracting  officers  were  neces¬ 
sary  to  ensure  that  we  understood  and  captured  all  the  important 
goods  and/or  services  in  an  action.  The  challenges  we  faced  and  how 
we  handled  them  in  our  analysis  are  described  next. 


Activities  That  Were  Not  Well  Described  by  Existing 
PSC  Codes 

We  had  trouble  identifying  appropriately  descriptive  codes  for  two 
categories  of  activities:  (1)  studies  performed  as  part  of  professional 
services,  which,  in  our  opinion,  required  more-descriptive  codes  to 
better  capture  the  specific  nature  of  the  study  being  performed  and 
(2)  warranties,  for  which  no  PSC  currently  exists.  In  each  of  these 
cases,  we  created  new  codes  for  our  analyses. 
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Studies.  In  our  analysis,  we  identified  three  major  types  of  stud¬ 
ies  performed  as  part  of  professional  services  by  contractors.  In 
each  case,  the  contracting  officer  had  assigned  a  Professional 
Services  PSC  (starting  with  R4),  perhaps  because  the  Special 
Studies  codes  (starting  with  B)  do  not  offer  an  appropriate  alter¬ 
native.  To  differentiate  among  them,  we  created  a  new  code  for 
each  of  the  following  types  of  studies: 

-  Support  for  day-to-day  analytic  needs  of  program  offices, 
e.g.,  administrative  support  for  collection  and  organization 
of  data  for  reports,  assistance  in  preparing  briefings,  and  so 
forth.  We  coded  these  activities  using  the  existing  four-digit 
codes  that  begin  with  R4. 

-  Engineering  studies  to  address  a  specific  problem,  e.g.,  how 
to  address  parts  obsolescence  (diminishing  sources  of  sup¬ 
ply)  problems,  the  effects  of  corrosion  on  the  structural  in¬ 
tegrity  of  an  airframe,  or  the  most  effective  repair  for  a  new 
maintenance  problem.  We  created  two  new  codes  for  these 
activities:  B560  for  special  studies  for  aircraft  and  B561  for 
special  studies  for  space  systems. 

-  Engineering  studies  to  support  the  design  and/or  installa¬ 
tion  of  a  modification.  Because  codes  with  the  prefix  KO  are 
used  for  modification  services  (for  example,  K010  is  the 
code  for  the  modification  of  weapons),  we  created  a  new 
code  with  the  prefix  K1  for  studies  that  support  such  modi¬ 
fications.  For  example,  for  a  study  done  in  preparation  for 
the  modification  of  a  weapon,  the  code  would  be  K1 10. 

Warranties  associated  with  purchased  parts  or  systems.  The 

closest  general  category  for  coding  a  warranty  was  Maintenance, 
Repair,  and  Rebuilding  of  Equipment  (J),  the  type  of  activity 
that  a  warranty  covers.  We  felt  that  it  would  be  useful  to  distin¬ 
guish  spending  on  actual  maintenance  and  repair  activities  from 
spending  on  what  is  in  effect  an  insurance  policy  for  problems 
with  parts  or  systems;  therefore,  we  created  the  code  prefix  J 1  to 
represent  warranties.  As  with  the  new  K1  prefix,  the  third  and 
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fourth  digits  of  the  PSC  denote  the  item  to  which  the  warranty 
applies. 

Activities  for  Which  Several  PSCs  Might  Be  Applicable 

We  also  encountered  several  types  of  activities  that  could  be  described 
by  more  than  one  PSC.  The  following  list  describes  the  activities  in 
this  category  that  we  encountered  most  frequently,  along  with  an  ex¬ 
planation  of  our  coding  decision.  It  is  likely  that  many  other  activities 
such  as  these  were  not  captured  in  our  sample. 

•  On-site  (e.g.,  at  a  depot)  engineering  support  from  a  contrac¬ 
tor.  Should  this  activity  be  considered  a  technical  representative 
service  (L)  or  some  type  of  engineering  support  (R4)?  In  our 
analysis,  we  coded  it  as  a  technical  representative  service  to  dif¬ 
ferentiate  it  from  the  program  office  support  discussed  earlier. 

•  Installation  of  a  modification  kit.  Should  this  activity  be  con¬ 
sidered  an  installation  service  (N)  or  a  modification  service  (K)  ? 
We  coded  it  as  a  modification  service  to  preserve  the  underlying 
intent  of  the  activity. 

•  Fleetwide  replacement  of  a  part  (e.g.,  a  vertical  stabilizer)  with 
a  similar  part  made  of  different  materials.  Should  this  activity 
be  considered  a  repair  activity  (J),  a  modification  activity  (K),  or 
a  parts  purchase  (goods  code)?  In  our  analysis,  we  attempted  to 
differentiate  modifications  from  repairs  based  on  whether  the 
nature  of  the  part  that  was  replaced  or  the  capabilities  of  the  sys¬ 
tem  changed  as  a  result  of  the  activity. 

•  Prototypes  for  research,  development,  test,  and  evaluation 
(RDT&E).  Should  prototypes  be  considered  part  of  the 
RDT&E  activity  (A)  or  considered  as  goods?  We  considered 
prototypes  to  be  part  of  the  RDT&E  if  they  were  changing  as 
the  RDT&E  progressed — i.e.,  if  they  were  part  of  the  RDT&E 
process. 

•  Production  follow-on  to  a  contract  that  was  originally  for  the 
purchase  of  RDT&E.  Should  this  activity  continue  to  be  coded 
as  an  RDT&E  activity  (A),  or  should  it  be  switched  to  a  produc- 
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tion  activity  (goods  code)?  For  our  analysis,  we  chose  the  latter 
option. 

•  Contractor  Logistics  Support  (CLS).  Should  this  activity  be 
considered  a  maintenance/repair  activity  (a  PSC  starting  with 
the  letter  J)  or  logistics  support  services  (R706)?  Because  PSCs 
beginning  with  R  are  generally  used  for  professional,  administra¬ 
tive,  and  management  support  services,  we  reasoned  that  codes 
beginning  with  J  were  more  descriptive  of  the  maintenance  and 
repair  activities  included  in  CLS. 

Actions  That  Include  Multiple  Activities 

As  discussed  in  Chapter  One,  the  DD350  form  forces  contracting 
officers  to  choose  a  single  PSC  for  each  contract  action.  For  contracts 
containing  multiple  activities,  it  may  be  difficult  to  identify  the  single 
best  PSC.  In  some  cases,  such  as  parts  and  services  required  for  a  re¬ 
pair,  the  multiple  activities  are  closely  affiliated.  In  other  cases,  such 
as  the  many  activities  required  to  maintain  a  facility,  the  activities  are 
clearly  distinct.  Examples  of  both  cases  are  listed  next.  In  each  in¬ 
stance,  a  single  PSC  cannot  capture  the  additional  activities  being 
purchased. 

Closely  Affiliated  Activities 

•  Modification  kit  purchase  (goods  code)  plus  a  warranty  for  the 
kit  (the  J 1  code  prefix  created  for  this  analysis) 

•  Test  equipment  purchase  (goods  code)  plus  maintenance  of  the 
equipment  (J) 

•  Repair  activity  (J)  that  includes  contractor-provided  replacement 
parts  (goods  code) 

•  Systems  integration  services  to  design  the  installation  of  a  modi¬ 
fication  kit  (the  K1  code  prefix  created  for  this  analysis),  pur¬ 
chase  of  the  kit  (goods  code),  and  installation  of  the  kit  (KO) 

•  Total  system  support  that  includes  maintenance  of  an  aircraft  (J) 
and  software  (D),  systems  engineering  services  to  support  the 
program  office  (R4),  and  personnel  training  (U) 
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•  Operation  and  maintenance  of  prepositioned  supplies  and  facili¬ 
ties  that  include  the  lease  or  rental  of  facilities  (X),  facility  main¬ 
tenance  (S),  and  maintenance  of  equipment  (J). 

Multiple  Distinct  Activities 

•  Several  installation  support  services,  e.g.,  housekeeping  (S)  and 
building  maintenance  (Z) 

•  Charter  airlift  for  people  (V221)  and  cargo  (V121). 

In  our  analysis,  we  allowed  for  as  many  PSCs  as  necessary  to 
capture  the  range  of  purchased  activities.  We  then  identified  the 
highest-dollar  PSC  as  the  primary  PSC  for  the  action. 

Allocation  of  Expenditures  to  Multiple  PSCs 

One  of  the  issues  that  arose  in  assigning  multiple  PSCs  for  a  single 
contract  action  is  how  dollars  should  be  allocated  to  those  PSCs. 
During  our  interviews,  we  asked  contracting  officers  for  information 
on  the  dollar  amounts  associated  with  distinct  activities;  however, 
separate  dollar  amounts  for  those  activities  could  not  always  be  disen¬ 
tangled. 

In  such  cases,  we  designated  one  PSC  as  the  primary  code  for 
the  action  according  to  the  likely  size  of  its  share  of  the  obligation 
value.  We  then  employed  two  alternative  approaches  to  allocating 
dollars  among  the  PSCs  for  the  action.  One  approach  allocated  all 
dollars  to  the  primary  PSC  in  the  action,  reflecting  the  DD350’s  cur¬ 
rent  mode  of  assignment.  The  second  approach,  covering  the  other 
extreme,  spread  the  dollars  evenly  across  all  of  the  PSCs  in  the  action. 
Generally  speaking,  the  results  were  not  significantly  affected  by 
which  approach  was  used,  in  part  because  of  the  relative  infrequency 
of  multiple  PSCs  and  in  part  because  specific  types  of  activities  did 
not  systematically  qualify  for  the  primary  place  within  the  action. 


Research  Methodology  27 


Extrapolating  Results  to  the  Entire  DD350  Population 

Because  we  focused  on  larger-dollar-value  contract  actions  in  our  sur¬ 
vey  sample  and  because  of  the  different  sampling  rates  used  for  goods 
and  services,  our  survey  data  do  not  accurately  represent  the  charac¬ 
teristics  of  the  entire  population  of  Air  Force  FY02  DD350  actions. 
Therefore,  we  reweighted  the  survey  results  to  accurately  reflect  the 
true  mix  of  contract  actions  and  dollars  in  the  entire  FY02  DD350 
population  before  drawing  conclusions  about  the  accuracy  and  ade¬ 
quacy  of  these  data.  In  the  next  chapter,  we  indicate  when  our  find¬ 
ings  are  based  on  reweighted  data  and  when  they  are  based  directly 
on  survey  data. 

For  the  reweighting  process,  we  first  distinguished  between  con¬ 
tract  actions  classified  in  the  DD350  as  goods  and  those  classified  as 
services.  We  then  identified  seven  different  contract- action  dollar-size 
categories.  These  categorizations  gave  us  a  total  of  14  distinct  action- 
type/dollar-size  strata — seven  for  goods  and  seven  for  services. 
Weights  were  applied  to  each  of  the  14  strata  as  appropriate.  Appen¬ 
dix  E  describes  our  reweighting  methodology  in  greater  detail. 


CHAPTER  THREE 

Findings 


In  this  chapter,  we  describe  our  findings  on  the  accuracy  of  DD350 
data  (how  closely  what  was  recorded  in  the  DD350  forms  matched 
what  we  learned  from  talking  to  the  contracting  officers  responsible 
for  filling  out  the  forms)  and  the  adequacy  of  the  data  (how  well  one 
PSC  described  what  was  purchased)  for  conducting  spend  analyses. 
As  discussed  in  the  previous  chapter,  we  reweighted  the  survey  data  to 
reflect  the  broader  population  of  Air  Force  FY02  DD350  data. 


Accuracy  of  the  DD350  Data 

DD350  data  have  several  inherent  limitations  that  might  affect  their 
accuracy.  The  current  set  of  PSCs  is  meant  to  be  used  by  all  organiza¬ 
tions  in  the  federal  government.  To  be  general  enough  to  serve  this 
purpose,  the  PSCs  may  not  be  sufficient  to  fully  characterize  the 
range  of  services  purchased  by  the  Air  Force  and  by  the  DoD  more 
generally.  Thus,  as  we  discussed  in  Chapter  Two,  assigning  an  appro¬ 
priate  PSC  can  be  difficult.  In  addition,  because  only  one  PSC  can  be 
assigned  to  an  action,  the  primary  activity  in  the  contract  must  be 
chosen.  This  implies  that  spending  on  some  categories  of  goods  or 
services  might  be  understated  or  overstated. 

For  example,  consider  two  hypothetical  contract  actions  used  to 
purchase  janitorial  and  food  services — two  services  with  different 
PSCs.  Suppose  the  first  action  purchased  $10  of  janitorial  services 
and  $5  of  food  services  (and  would  be  reported  as  a  $15  “janitorial” 


29 


30  An  Assessment  of  Air  Force  Data  on  Contract  Expenditures 


contract),  while  the  second  action  purchased  $10  of  janitorial  services 
and  $15  of  food  services  (and  would  be  reported  as  a  $25  “food  serv¬ 
ices”  contract).  The  total  spent  on  each  of  the  two  types  of  services  is 
$20,  but  the  reported  spending  for  janitorial  services  would  be  $15 
(an  understatement),  and  the  reported  spending  for  food  services 
would  be  $25  (an  overstatement).  A  similar  problem  exists  with  pur¬ 
chases  of  goods  that  might  include  services  (such  as  warranties)  or 
purchases  of  services  that  might  include  the  purchase  of  a  good.  In 
either  case,  using  only  one  PSC  obscures  information  that  helps  to 
describe  the  purchase  more  accurately. 

In  addition  to  coding  limitations,  we  were  told  during  back¬ 
ground  interviews  conducted  during  this  study  that  although  con¬ 
tracting  officers  receive  training  in  how  to  complete  and  submit  a 
DD350  form,  they  do  not  receive  specific  training  in  how  to  deter¬ 
mine  the  appropriate  PSC  for  an  action,  and  they  have  considerable 
discretion  in  assigning  codes.  Standardization  in  coding  therefore  is 
limited,  and  this  problem  is  complicated  by  the  fact  that  there  is  very 
little  auditing  of  how  DD350  information  is  recorded. 

Another  problem  that  potentially  affects  the  accuracy  of  DD350 
information  is  that  filling  out  the  form  is  considered  by  contracting 
officers  to  be  a  final  formality  in  completing  a  contract  action.  Many 
Air  Force  personnel  we  surveyed  indicated  that  the  DD350  system  is 
difficult  to  maneuver.  The  best  characterization  of  the  purchased 
goods  or  services  may  not  always  satisfy  system  input  requirements. 

The  difficulties  with  DD350  coding  collectively  underlie  the  re¬ 
sults  shown  in  Table  3.1.  In  our  analysis,  we  define  accuracy  as  a 
match  between  the  first  character  of  the  PSC  found  in  the  DD350 
record  and  the  first  character  of  the  primary  RAND-assigned  PSC. 
(When  we  assigned  multiple  PSCs  to  a  contract  action,  the  primary 
RAND-assigned  PSC  is  the  PSC  accounting  for  the  most  dollars.) 
Thus,  we  examined  how  accurately  the  DD350  is  capturing  the  pri¬ 
mary  PSC.  Under  this  generous  definition  of  accuracy,  our 
reweighted  survey  results  indicate  that  50  percent  of  the  recorded  Air 
Force  FY02  DD350  actions  are  inaccurate  in  terms  of  their  descrip- 
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tion  of  the  purchased  activities,  and  these  inaccuracies  represent  39 
percent  of  the  contract  dollars.1 

Details  of  DD350  Coding  Inaccuracies 

To  further  explore  the  nature  of  DD350  PSC  inaccuracies,  we 
grouped  contract  actions  into  the  categories  shown  in  Figure  3.1.  Ac¬ 
tions  for  which  the  first  character  of  the  primary  RAND-assigned 
PSC  and  first  character  of  the  DD350  PSC  matched  are  represented 
by  the  black  oval  in  the  figure.  If  the  two  PSCs  did  not  match  at  the 
first  character,  we  next  determined  whether  they  were  both  of  the 
same  type  (that  is,  either  goods  or  services).  The  DD350  contract  ac¬ 
tions  that  we  confirmed  were  accurately  coded  as  a  good  or  a  service, 
through  our  interviews  with  contracting  officers,  are  represented  by 
the  lighter-shaded  oval  in  the  figure.  For  the  remaining  actions,  our 
PSC  assignment  differed  from  the  type  of  PSC  recorded  in  the 
DD350  database.  Actions  recorded  in  the  DD350  as  services  that  we 
determined  through  our  interviews  were  best  described  as  goods,  or 
actions  recorded  as  goods  that  we  determined  were  best  described  as 
services,  are  represented  by  the  two  darker-shaded  ovals  in  the  figure. 

Table  3.1 

Estimated  Accuracy  of  PSCs  for  Describing  Air  Force  FY02  DD350  Contract 
Actions 


DD350  PSC  Is  Accurate3 

DD350  PSC  Is  Not  Accurate 

Contract  Actions 

50% 

50% 

Dollars  Represented  by 

61% 

39% 

Contract  Actions 

NOTE:  Percentages  are  based  on  survey  data  that  has  been  reweighted  to  reflect  all  Air 
Force  FY02  DD350  contract  actions. 

a  Accurate  refers  to  a  match  between  the  first  character  of  a  DD350  PSC  and  the  first 
character  of  a  primary  RAND-assigned  PSC. 


1  The  findings  from  our  survey  sample  (see  Chapter  Two)  have  been  extrapolated  to  the 
population  of  Air  Force  FY02  DD350  contract  actions. 
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The  figures  and  tables  that  follow  use  the  framework  in  Figure 
3.1  to  examine  the  nature  of  the  coding  errors  in  the  Air  Force  FY02 
DD350  population  that  we  identified  through  our  interviews. 

As  shown  in  Table  3.1,  we  estimate  that  39  percent  of  the  con¬ 
tract  dollars  in  the  Air  Force  FY02  DD350  database  have  inaccurate 
PSC  information.  Figure  3.2  illustrates  how  this  percentage  is  broken 
out:  Of  the  39  percent,  16  percent  of  the  expenditures  are  for  actions 
that  are  correctly  coded  as  a  good  or  service,  but  the  first  character  of 
the  primary  RAND-assigned  PSC  corresponds  to  a  different  good  or 
service  than  the  one  recorded  in  the  DD350  form.  The  other  23  per¬ 
cent  of  these  dollars  is  associated  with  actions  that  are  miscoded  as  a 
good  or  service.  Breaking  down  the  23  percent  further,  1 8  percent  is 
for  goods  that  should  have  been  coded  as  services,  and  the  other 
5  percent  is  for  services  that  should  have  been  coded  as  goods. 

Contract  Actions  for  Goods.  Figure  3.3  shows  how  coding  in¬ 
accuracies  for  actions  recorded  as  goods  in  the  DD350  database  vary 
with  the  dollar  value  of  the  action. 

Figure  3.1 

Hierarchy  of  DD350  Coding  Inaccuracies 


All  contract  actions 
in  sample 


RAND  MG274-3. 1 
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Figure  3.2 

Estimated  Accuracy  of  PSC  Codes  by  Amount  of  Expenditures  in  Air  Force 
FY02  DD350  Database,  All  Actions 


I  First  character  of  DD350  PSC  and 
RAND-assigned  PSC  match 

]  Correctly  coded  as  good  or  service, 
but  first  character  of  DD350  PSC 
and  RAND-assigned  PSC  do  not 
match 

]  Incorrectly  coded  as  good  or  service 


RAND  MG274-3.2 


NOTES:  Percentages  are  based  on  survey  data  that  have  been  reweighted  to  reflect 
the  entire  population  of  Air  Force  FY02  DD350  contract  actions.  Value  of  all  contract 
actions  totals  $49  billion. 


For  goods  actions  worth  less  than  $1  million,  we  estimate  that 
87  percent  of  the  dollars  should  have  been  identified  as  expenditures 
for  services.  Actions  in  this  dollar  category  of  goods  actions  account 
for  only  6  percent  of  the  total  Air  Force  FY02  DD350  dollars  ($2.7 
billion).  For  goods  actions  worth  between  $1  million  and  $50  mil¬ 
lion,  which  account  for  21  percent  of  the  DD350  dollars  ($10.2  bil¬ 
lion),  approximately  half  the  dollars  should  have  been  recorded  as 
being  for  services.  Goods  actions  valued  at  greater  than  $50  million 
account  for  22  percent  of  DD350  dollars  ($10.9  billion)  and  fare  bet¬ 
ter  in  our  analysis  of  accuracy:  Only  13  percent  of  those  dollars 
should  have  been  coded  as  services.  These  results  run  counter  to  what 
we  had  expected.  We  had  anticipated  that  small  goods  actions  would 
be  accurately  coded,  but  that  large  goods  actions  might  include  the 
purchase  of  a  variety  of  items,  and  therefore  would  be  relatively  diffi¬ 
cult  to  code  accurately  with  only  one  PSC. 
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Figure  3.3 

Estimated  Coding  Inaccuracies  for  Air  Force  FY02  DD350  Actions  Recorded  as 
Goods,  by  Dollar  Value  of  Contract  Action 


Goods  <$1  million  Goods  $1  million-$50  million  Goods  >$50  million 

$2.7  billion  (6%)  $10.2  billion  (21%)  $10.9  billion  (22%) 


/ 

87% 


|  First  character  of  PSC  correctly  coded 

]  Correctly  coded  as  a  good,  but  wrong 
type  of  good 

]  Incorrectly  coded  as  a  good 
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NOTE:  Percentages  are  based  on  survey  data  that  have  been  reweighted  to  reflect  the 
entire  population  of  Air  Force  FY02  DD350  contract  actions. 

Following  are  some  examples  of  DD350  actions  that  were  incor¬ 
rectly  coded  as  goods  rather  than  services: 

•  An  action  coded  in  the  DD350  database  as  aircraft  goods  was 
primarily  for  installation  of  commercial  engines  (re-engining), 
plus  designs  and  kits  for  reliability  improvements  to  various  air¬ 
craft  parts  (such  as  door  seals  and  drain  lines). 

•  An  action  coded  as  satellite  goods  was  primarily  for  R&D  for 
satellite  system  modernization. 
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•  An  action  coded  as  computer  hardware  goods  was  primarily  for 
developing  computer  system  software  and  converting  data  to  the 
new  system. 

Table  3.2  provides  additional  insight  into  how  selected  contract 
actions  valued  between  $1  million  and  $50  million  that  were  coded 
as  goods  in  the  DD350  database  should  be  re-coded,  based  on  our 
analysis.  The  table  reflects  our  survey  sample  rather  than  the  survey 
data  extrapolated  to  the  entire  Air  Force  FY02  DD350  database. 

The  rows  in  Table  3.2  group  expenditures  by  the  first  character 
of  the  PSC  recorded  in  the  DD350  database.  The  columns  show  the 
percentages  of  expenditures  in  each  row  that  fall  into  each  of  the  in¬ 
dicated  RAND-assigned  PSC  categories.  The  boldface  numbers  in 
this  table  (and  in  Table  3.3)  represent  percentages  of  expenditures  for 
which  the  first  character  of  the  primary  RAND-assigned  PSC  and 
DD350  PSC  matched. 

Using  the  first  row  of  Table  3.2  as  an  example,  17  percent  of  the 
expenditures  in  our  sample  that  were  recorded  in  the  DD350  forms 
with  a  PSC  starting  with  the  number  1  were  accurately  coded — that 
is,  we  also  assigned  primary  PSCs  whose  first  character  was  the  num¬ 
ber  1  to  those  actions.  Our  analyses  indicate  that  the  remainder 
of  these  obligations  should  have  been  recorded  as  various  types  of 

Table  3.2 

Percentage  of  Expenditures  in  Survey  Sample  Reassigned  to  Other  PSCs, 
Goods  Actions  Between  $1  Million  and  $50  Million 


RAND-Assigned  PSCs 


Goods 

Services 

Total 

1 

2 

5 

Other 

A  D 

J 

K 

M 

R  Other 

DD350 

1  17 

18  4 

26 

18 

4 

1  13 

100 

PSCs 

2 

71 

3 

22 

5 

100 

5 

68 

12 

20 

100 

NOTE:  Percentages  are  based  on  unweighted  survey  data. 
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services.  For  example,  18  percent  should  have  been  recorded  as  R&D 
(A),  4  percent  should  have  been  recorded  as  an  automatic  data  proc¬ 
essing  and  telecommunications  service  (D),  26  percent  should  have 
been  recorded  as  a  service  related  to  maintenance,  repair,  and  re¬ 
building  of  equipment  (J),  and  so  forth. 

Table  3.2  illustrates  that  many  actions  between  $1  million  and 
$50  million  that  were  recorded  as  goods  were  actually  for  R&D  (A), 
maintenance  (J),  and  modifications  (K).  One  possible  explanation  for 
these  discrepancies  is  that  after  a  good  is  purchased,  follow-on  actions 
for  services  or  ancillary  activities  related  to  the  initial  purchase  of  the 
good  somehow  retain  the  original  PSC  and  thus  escape  detection  as  a 
service  in  the  official  records. 

Contract  Actions  for  Services.  Figure  3.4  shows  how  our  esti¬ 
mates  of  the  coding  inaccuracies  for  actions  recorded  as  services  in  the 
DD350  database  vary  with  the  dollar  value  of  the  action. 

Interestingly,  the  pattern  illustrated  in  Figure  3.4  is  almost  the 
reverse  of  that  in  Figure  3.3.  For  service  actions  worth  less  than  $1 
million  (which  made  up  14  percent  of  Air  Force  FY02  DD350  dol¬ 
lars),  we  estimate  that  only  1  percent  should  have  been  recorded  as 
goods  actions.  For  actions  worth  between  $1  million  and  $50  mil¬ 
lion,  we  estimate  that  only  5  percent  of  expenditures  should  have 
been  recorded  as  goods  actions,  although  43  percent  of  the  expendi¬ 
tures  should  have  been  coded  as  different  services  than  what  was  indi¬ 
cated  in  the  DD350  database.  The  coding  for  the  largest  services  ac¬ 
tions  (those  for  more  than  $50  million)  is  the  most  inaccurate:  We 
estimate  that  48  percent  of  the  services  expenditures  should  have  been 
coded  as  goods. 

Following  are  examples  of  DD350  service  actions  that  were  used 
to  purchase  goods: 

•  A  satellite  system  R&D  action  that  included  initial  production 
activities 

•  A  systems  engineering  action  that  included  depot  spare  parts 
replenishment. 
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Figure  3.4 

Estimated  Coding  Inaccuracies  for  Air  Force  FY02  DD350  Actions  Recorded 
as  Services,  by  Dollar  Value  of  Contract  Action 


Services  <$1  million 
$6.8  billion  (14%) 

1% 


13%  “\ 


86% 


Services  $1  million-$50  million  Services  >$50  million 
$14.1  billion  (29%)  $4.3  billion  (9%) 


3% 


|  First  character  of  PSC  correctly  coded 

]  Correctly  coded  as  a  service,  but  wrong 
type  of  service 

|  Incorrectly  coded  as  a  service 
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NOTE:  Percentages  are  based  on  survey  data  that  have  been  reweighted  to  reflect  the 
entire  population  of  Air  Force  FY02  DD350  contract  actions. 

Some  examples  of  actions  that  were  correctly  coded  as  services 
but  would  be  better  described  by  a  PSC  other  than  the  one  listed  in 
the  DD350  database  include  the  following: 

•  An  equipment  modification  action  that  was  primarily  for  educa¬ 
tion  and  training  services 

•  A  professional  services  action  that  included  the  testing  and  inte¬ 
gration  of  new  sensors  for  aircraft 

•  A  professional  services  contract  that  covered  engineering  work  to 
develop  a  kit  for  installing  new  communications  equipment. 
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Tables  3.3  and  3.4  show  the  percentage  of  expenditures  for  con¬ 
tract  actions  identified  in  the  DD350  as  services  that  were  reassigned 
to  other  PSCs. 

Table  3.3  highlights  the  major  reassignments  for  contract  ac¬ 
tions  between  $1  million  and  $50  million  identified  as  services  in  the 
DD350  database.  For  example,  2  percent  of  the  expenditures  identi¬ 
fied  with  a  PSC  beginning  with  “R”  in  the  DD350  (professional, 
administrative,  and  management  support  services)  should  have  been 
recorded  as  goods  with  a  PSC  that  begins  with  1 .  An  R&D  service 
code  (A)  would  have  been  better  for  3  percent  of  these  dollars,  and  a 
modification-of-equipment  service  code  (K)  would  have  been  better 
for  32  percent  of  the  dollars.  Only  19  percent  of  these  dollars  were 
correctly  coded  with  the  letter  R — that  is,  we  also  assigned  a  primary 
PSC  beginning  with  the  letter  R  to  these  actions. 

Table  3.4  highlights  important  PSC  shifts  for  service  contract 
actions  for  more  than  $50  million.  It  shows  that  only  52  percent  of 
the  obligations  for  contract  actions  for  more  than  $50  million  that 
were  coded  in  the  DD350  database  as  R&D  services  (A)  were  accu¬ 
rately  coded.  Our  analyses  indicate  that  a  large  percentage  of  these 
expenditures  was  actually  used  to  purchase  goods.  It  is  possible  that 
this  discrepancy  results  from  production  activities  that  were  the  result 
of  what  initially  were  research  and  development  projects. 


Adequacy  of  the  DD350  Data 

As  explained  in  the  previous  chapter,  the  PSC  used  to  describe  con¬ 
tract  actions  consists  of  four  characters.  The  PSC  manual  classifies 
actions  coded  with  PSCs  that  begin  with  a  letter  as  services;  codes 
that  begin  with  numbers  correspond  to  goods.  The  other  three  char¬ 
acters  of  the  PSC  provide  more  detailed  information  about  the  service 
or  good  being  purchased.  We  used  a  broad  definition  of  the  adequacy 
of  using  a  single  PSC  to  describe  a  purchase:  If  more  than  one  good 
or  service  was  included  in  an  action,  and  each  of  them  could  be  de¬ 
scribed  with  codes  that  have  the  same  first  character,  we  considered  a 


Table  3.3 

Percentage  of  Expenditures  in  Survey  Sample  Reassigned  to  Other  PSCs,  Service  Actions  Between  $1  Million 
and  $50  Million 


RAND-Assigned  PSCs 

Goods  Services 


1 

5 

A 

B 

D 

J 

K 

L  M 

R 

S 

U 

Other 

Total 

A 

6 

6 

72 

2 

1 

12 

1 

100 

J 

6 

94 

100 

DD350 

K 

8 

36 

7 

6 

42 

100 

PSCs 

R 

2 

3 

7 

4 

15 

32 

2 

19 

13 

3 

100 

S 

70 

30 

100 

Other 

100 

100 

NOTE:  Percentages  are  based  on  unweighted  survey  data. 
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Table  3.4 

Percentage  of  Expenditures  in  Survey  Sample  Reassigned  to  Other  PSCs, 
Service  Actions  for  More  Than  $50  Million 


RAND-Assigned  PSCs 


Goods  Services 


1  2  5  A  K  L  Total 


DD350  A  24  16  3  52  4  1  100 

PSCs 


Note:  Percentages  are  based  on  unweighted  survey  data. 


single  PSC  to  be  adequate.  Differences  in  the  second,  third,  and 
fourth  digits  of  the  PSC  were  ignored.  Note  that  this  is  a  very  gener¬ 
ous  definition  of  adequacy — e.g.,  maintenance  and  repair  services  be¬ 
gin  with  the  letter  J  but  could  cover  anything  from  maintenance  of 
nuclear  ordnance  (code  JO  1 1 )  to  upkeep  of  live  animals  (code  J088). 

Table  3.5  summarizes  our  results  for  adequacy,  which  are  ex¬ 
trapolated  from  our  survey  sample  to  the  Air  Force  FY02  DD350 
population.  For  89  percent  of  the  contract  actions,  one  PSC  was  ade¬ 
quate  to  describe  what  was  purchased.  However,  these  actions  repre¬ 
sented  only  73  percent  of  the  DD350  dollars.  Contract  actions  for 
which  one  PSC  was  inadequate  to  fully  describe  the  services  and/or 
goods  represented  27  percent  of  the  dollars  included  in  the  Air  Force 
FY02  DD350  data,  or  just  under  $14  billion. 

Table  3.5 

Estimated  Adequacy  of  a  Single  PSC  to  Describe  FY02  Air  Force  Contract 
Actions  in  the  DD350  Database 


Single  PSC 

Single  PSC  Not 

Adequate 

Adequate3 

Total 

Contract  Actions  (%) 

89 

11 

100 

Contract  Action  Dollars  (%) 

73 

27 

100 

aMultiple  PSCs  with  differing  first  characters  are  needed  to  describe  the  goods  and/or 
services  purchased  by  these  contract  actions. 

NOTE:  Percentages  are  based  on  survey  data  that  have  been  reweighted  to  reflect  the 
entire  population  of  Air  Force  FY02  DD350  contract  actions. 
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The  following  are  examples  of  contract  actions  identified  during 
our  interviews  that  involved  activities  of  more  than  one  distinct  type: 

•  A  radar  purchase  (a  good)  that  included  some  support  services 
for  installation 

•  An  aircraft  purchase  (a  good)  that  included  logistics  and  training 
support  (a  service) 

•  Services  for  locating  nonstandard  parts  that  included  costs  for 
purchasing  parts  (goods). 

In  Figure  3.5,  we  examine  the  adequacy  of  a  single  PSC  for  ac¬ 
tions  that  fell  within  three  dollar-size  categories:  less  than  $  1  million, 
between  $1  million  and  $50  million,  and  greater  than  $50  million. 
The  figure  shows  the  percentage  of  obligations  in  each  dollar-size 
category  (plus  confidence  intervals  on  those  estimates)  associated  with 
contract  actions  that  require  more  than  one  PSC  to  fully  describe  the 
purchase. 

Figure  3.5  indicates  that  larger  actions  are  indeed  more  likely  to 
include  expenditures  for  multiple  activities.  The  figure  shows  that  35 
percent  of  the  dollars  for  actions  worth  more  than  $50  million  and 
30  percent  of  the  dollars  for  actions  valued  between  $1  million  and 
$50  million  are  for  contract  actions  that  require  multiple  PSCs.  By 
comparison,  only  7  percent  of  the  dollars  for  actions  worth  less  than 
$1  million  require  multiple  PSCs. 

We  next  examine  whether  certain  categories  of  goods  or  services 
actions  are  more  likely  to  contain  multiple  activities.  In  Figure  3.6, 
we  illustrate  the  adequacy  of  a  single  PSC  for  actions  that  fall  within 
five  PSC  groupings  according  to  the  DD350  data:  R&D,  other 
services,  aircraft  and  components,  engines  and  components,  and 
other  goods.2  As  in  Figure  3.5,  Figure  3.6  shows  the  estimated 


2  The  R&D  category  covers  PSCs  beginning  with  the  letter  A.  The  other  services  category 
covers  PSCs  beginning  with  letters  other  than  A.  The  aircraft  and  components  category  cov¬ 
ers  PSCs  beginning  with  a  1 .  The  engines  and  components  category  covers  PSCs  beginning 
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Figure  3.5 

Estimated  Adequacy  of  Using  Only  a  Single  PSC  to  Describe  Air  Force  FY02 
DD350  Contract  Actions,  by  Dollar  Size  of  Contract  Action 


Size  of  contract  action 


RAND  MG274-3.5 

NOTES:  Percentages  are  based  on  survey  data  that  have  been  reweighted  to  reflect 
the  entire  population  of  Air  Force  FY02  DD350  contract  actions.  Dashed  vertical  lines 
represent  95-percent  confidence  intervals. 

percentages  of  Air  Force  FY02  DD350  contract  dollars  that  are  made 
by  contract  actions  that  require  more  than  one  PSC  to  fully  describe 
the  purchase. 

Figure  3.6  illustrates  that  even  though  the  criteria  we  used  to  de¬ 
termine  whether  multiple  PSCs  are  required  are  fairly  loose  (the  PSCs 
must  differ  at  the  first  character),  the  need  for  multiple  PSCs  is  wide¬ 
spread.  For  example,  our  analyses  indicate  that  29  percent  of  the 


with  a  2.  The  other  goods  category  covers  PSCs  beginning  with  a  number  other  than  1  or  2. 
See  Appendix  B  for  further  information. 
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Figure  3.6 

Estimated  Adequacy  of  Using  Only  a  Single  PSC  to  Describe  Air  Force  FY02 
DD350  Contract  Actions,  by  PSC  Category 
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R&D  Other  services  Aircraft  and  Engines  and  Other  goods 

components  components 

Total  value  $9.1  billion  $15.4  billion  $16.3  billion  $2.5  billion  $5.0  billion 

Type  of  good  or  service  purchased 

RAND  MG274  3.6 


NOTES:  Percentages  are  based  on  survey  data  that  have  been  reweighted  to  reflect 
the  entire  population  of  Air  Force  FY02  DD350  contract  actions.  Dashed  vertical  lines 
represent  95-percent  confidence  intervals.  There  is  no  confidence  interval  for  engines 
and  components  because  the  variance  of  the  estimate  is  zero. 


dollars  within  the  “aircraft  and  components”  category  required  mul¬ 
tiple  PSCs  to  be  fully  described.  Obligations  in  the  “engines  and 
components”  category  proved  to  be  the  sole  exception:  none  of  the 
actions  in  this  category  required  multiple  PSCs.  These  actions  are 
generally  for  subsystems  and  specific  items,  which  involve  smaller 
dollar  amounts  that  are  less  likely  to  involve  multiple  activities. 

Figures  3.5  and  3.6  and  Table  3.5  raise  the  question  of  how 
much  more  information  could  be  “captured”  by  the  DD350  if  the 
form  allowed  the  inclusion  of  more  than  one  PSC  (with  correspond¬ 
ing  dollar  amounts)  to  describe  a  contract  action.  We  found  that  even 
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for  those  contract  actions  for  which  more  than  one  PSC  was  neces¬ 
sary,  the  primary  PSC  (the  one  that  we  determined  described  the 
most  contract  dollars  covered  by  the  action)  accounted  for  60  to  80 
percent  of  the  dollars  in  the  action  (based  on  unweighted  survey 
data).3  Considering  Air  Force  FY02  DD350  data  as  a  whole,  allowing 
only  one  PSC  to  describe  contract  actions  prevents  an  accurate  char¬ 
acterization  of  5  to  11  percent  of  the  total  contract  dollars,  or  roughly 
$2  billion  to  $5  billion. 

As  previously  discussed,  the  existing  set  of  PSCs  did  not  allow  us 
to  identify  certain  types  of  purchases,  such  as  warranties  and  specific 
kinds  of  engineering  studies.  Introducing  new  codes  for  these  activi¬ 
ties  enabled  us  to  better  identify  5  to  6  percent  of  the  total  contract 
dollars.  Using  only  a  single  PSC  per  action,  these  additional  codes 
identified  about  5  percent  of  the  dollars.  If  multiple  PSCs  were  al¬ 
lowed,  these  additional  codes  identified  6  percent  of  dollars.  We  re¬ 
turn  to  these  issues  in  the  next  chapter. 


Implications  for  Tracking  Service  Expenditures 

The  coding  problems  described  in  this  chapter  lead  to  the  conclusion 
that  FY02  DD350  data  on  Air  Force  expenditures  for  goods  and 
services  present  a  somewhat  misleading  picture  of  actual  expenditures. 
Figure  3.7  shows  the  differences  between  the  mix  of  goods  and 
services  purchased  by  the  Air  Force  according  to  DD350  data  on  con¬ 
tract  actions  and  the  mix  of  goods  and  services  based  on  our  survey 
data.  The  RAND  estimates  allow  for  both  more  accurate  coding  of 


3  The  upper  and  lower  bounds  for  this  range  are  due  to  different  treatments  of  actions  for 
which  contracting  officers  were  unable  to  break  out  the  dollars  associated  with  distinct  goods 
and  services  purchased  through  the  contract  action.  The  lower  end  of  this  range  results  when 
expenditures  are  evenly  spread  across  all  the  PSCs  identified  for  a  component  of  the  contract 
actions,  where  multiple  PSCs  are  needed  to  describe  the  goods  or  services,  but  the  contract¬ 
ing  officer  was  not  able  to  assign  dollars  to  each  of  the  PSCs.  The  upper  end  of  the  interval 
results  when  expenditures  are  attributed  to  the  primary  PSC  for  that  part  of  the  contract 
action. 
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Figure  3.7 

Share  of  Overall  Air  Force  FY02  DD350  Expenditures  by  Purchase  Category, 
RAND  Estimates  Versus  DD350  Data 


components  components 

RAND  MG274-3.7 

NOTES:  Percentages  are  based  on  survey  data  that  have  been  reweighted  to  reflect 
the  entire  population  of  Air  Force  FY02  DD350  contract  actions.  Dashed  vertical  lines 
represent  95-percent  confidence  intervals. 


the  primary  PSC  and  the  option  of  using  multiple  PSCs  to  charac¬ 
terize  a  contract  action.  The  bars  in  the  figure  show  the  shares  of 
overall  Air  Force  FY02  DD350  expenditures  for  various  categories  of 
purchases,  based  on  DD350  PSCs.  The  horizontal  dashes  represent 
the  distribution  of  purchases  based  on  our  interviews  with  contract¬ 
ing  officers,  and  the  dotted  vertical  lines  indicate  the  95-percent  con¬ 
fidence  interval  for  the  estimates. 

The  two  categories  for  which  there  are  significant  differences  be¬ 
tween  our  estimates  and  the  DD350  data  are  other  services  and  air¬ 
craft  and  components.  We  found  that  a  much  higher  share  of  other 
services  was  purchased  than  is  indicated  by  the  DD350  data:  45  per¬ 
cent  according  to  our  estimates  as  opposed  to  only  31  percent 
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according  to  the  DD350  data.  We  found  that  fewer  dollars  went  to 
the  aircraft  and  components  category  than  were  recorded  in  the 
DD350  database — the  DD350  data  indicated  that  33  percent  of 
spending  went  to  this  category,  while  our  survey  results  led  us  to  con¬ 
clude  that  the  actual  share  was  only  21  percent.4  On  the  other  hand, 
the  share  of  expenditures  for  R&D,  engines  and  components,  and 
other  goods  appears  to  be  accurately  represented  by  DD350  data. 

The  end  result  of  the  differences  between  DD350  data  and  the 
RAND  estimates  is  shown  in  Figure  3.8. 

Figure  3.8 

Percentage  of  Air  Force  Spending  on  Services,  RAND  Estimate  Versus  Air 
Force  FY02  DD350  Data 
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NOTES:  Percentages  are  based  on  survey  data  that  have  been  reweighted  to  reflect  entire 
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dence  interval. 


□  DD350  data 
—  RAND  estimates 


1-  66% 


51% 


Share  of  spending  on  services 


4  The  differences  for  other  services  and  aircraft  and  components  are  statistically  significant, 
because,  as  shown  in  Figure  3.7,  the  confidence  intervals  for  these  two  categories  do  not 
contain  the  percentage  of  expenditures  that  is  derived  from  the  DD350  database. 
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Figure  3.8  shows  that,  according  to  Air  Force  FY02  DD350 
data,  services  accounted  for  5 1  percent  of  contract  purchases.  We  es¬ 
timate  that  the  actual  percentage  of  spending  on  services  is  greater — 
66  percent,  as  represented  by  the  horizontal  dash  in  the  figure.5  Once 
again,  the  dotted  vertical  line  represents  a  95 -percent  confidence  in¬ 
terval,  and  the  difference  between  the  RAND  estimate  and  percent¬ 
age  based  on  DD350  data  is  statistically  significant. 


5  As  noted  in  Chapter  Two,  there  were  two  approaches  available  for  allocating  dollars  to 
actions  requiring  multiple  PSCs  when  the  contracting  officer  was  unable  to  provide  an  allo¬ 
cation.  One  approach  was  to  allocate  all  dollars  to  the  primary  PSC,  and  the  other  approach 
was  to  spread  the  dollars  evenly  across  all  the  PSCs.  Our  estimate  of  the  percentage  of  expen¬ 
ditures  for  services,  shown  in  Figure  3.8,  is  based  on  the  first  approach.  We  found  that  the 
approach  used  to  allocate  dollars  requiring  multiple  PSCs  made  little  difference  in  the 
estimate. 


CHAPTER  FOUR 


Summary  and  Policy  Recommendations 


In  this  chapter,  we  summarize  our  findings  from  this  study,  discuss  in 
further  detail  potential  causes  of  problems  with  DD350  data,  and 
recommend  actions  the  Air  Force  could  take  in  the  short  term  and 
long  term  to  help  improve  the  usefulness  of  DD350  data  for  future 
spend  analyses. 


Summary  of  Findings 

Table  4.1  summarizes  our  most  important  findings  on  the  accuracy 
and  adequacy  of  DD350  data  for  conducting  spend  analyses.  We 
conclude  that  analyses  of  DD350  data  provide  useful  information 
about  the  nature  of  the  Air  Force’s  expenditures. 

However,  the  Air  Force  should  understand  that  (1)  DD350- 
based  analyses  will  likely  underrepresent  the  range  of  goods  and 
services  purchased  because  of  limited  information  from  a  single  PSC 
and  (2)  these  analyses  will  likely  be  distorted  by  miscodings,  particu¬ 
larly  for  several  categories  of  services.  Supplemental  data  are  needed 
to  develop  a  detailed  understanding  of  the  types  of  goods  and  services 
the  Air  Force  purchases  and  to  construct  appropriate  purchasing  and 
supply  management  strategies. 
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Table  4.1 

Summary  of  Findings  on  Adequacy  and  Accuracy  of  Air  Force  FY02 
DD350  Data 


Issue 


Findings 


Accuracy  of  PSC  coding  in 
DD350  database 

Adequacy  of  using  a  single  PSC 
in  DD350  database 


How  well  the  available  PSCs 
describe  Air  Force  purchases 


The  PSC  for  50  percent  of  contract  actions  (39  per¬ 
cent  of  contract  dollars)  is  coded  inaccurately. 

Services  are  undercounted. 

More  than  one  distinct  good  and/or  service  was  pur¬ 
chased  in  1 1  percent  of  contract  actions  (27  per¬ 
cent  of  contract  dollars). 

5  percent  to  1 1  percent  of  dollars  are  associated  with 
secondary  PSCs. 

Several  important  categories  of  activities  are  not 
fully  captured  in  the  current  codes. 

New  PSCs  could  be  used  to  better  identify  at  least 
5  percent  to  6  percent  of  contract  dollars. 


Potential  Reasons  for  Discrepancies 

The  results  of  our  analyses  raise  the  question  of  what  lies  behind  the 
pervasive  discrepancies  between  the  data  recorded  in  the  DD350 
forms  and  the  actual  goods  and/or  services  that  are  purchased.  During 
our  survey  interviews  and  subsequent  conversations  with  Air  Force 
contracting  officers,  we  learned  of  several  potential  factors  that  can 
lead  to  miscodings.  Some  explanations  for  the  discrepancies  are  hypo¬ 
thetical,  and  further  research  is  required  to  fully  understand  the  rele¬ 
vant  issues. 

First,  we  learned  firsthand  how  difficult  it  can  be  to  assign  the  right 
code  to  an  activity.  Chapter  Two  presents  several  examples  of  contract 
actions  for  which  a  number  of  different  PSCs  could  apply  and  cases 
in  which  there  appeared  to  be  no  available  PSC  to  describe  the  spe¬ 
cific  activity. 

Second,  it  may  be  that  some  PSCs  are  simply  being  carried  forward 
to  subsequent  actions  or  contracts,  even  though  the  goods  and/or 
services  being  purchased  are  changing  from  one  contract  action  to 
another.  For  example,  in  a  contract  for  aircraft  modification,  one  ac¬ 
tion  may  cover  engineering  support  to  design  the  installation  process, 
one  action  may  cover  the  modification  kits  themselves,  and  another 
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action  may  cover  installation.  Contracting  officers  may  end  up  using 
the  original  PSC  to  describe  each  of  these  distinct  activities.  As  an¬ 
other  example,  when  a  development  effort  transitions  into  the  pro¬ 
duction  of  prototypes  and  then  into  low-rate  initial  production,  con¬ 
tracting  officers  may  carry  forward  information  from  one  contract 
action  to  the  next  without  recoding  the  actions  to  reflect  the  change 
in  activities. 

Third,  contracting  officers  fill  out  the  DD350 forms,  but  they  may 
not  be  the  individtials  who  are  most  knowledgeable  about  the  exact  na¬ 
ture  of  the  goods  and/or  activities  being  purchased.  A  contracting  offi¬ 
cer’s  basic  understanding  of  a  good  or  service  may  not  be  sufficient  to 
enable  him  or  her  to  distinguish  from  among  many  potentially  rele¬ 
vant  PSCs.  Several  times  during  the  course  of  our  surveys,  the  con¬ 
tracting  officers  we  interviewed  brought  in  experts  to  help  with  tech¬ 
nically  complex  cases. 

Finally,  contracting  officers  view  the  DD350 form  as  the  final  for¬ 
mality  in  completing  a  contract  action.  Many  contracting  officers  told 
us  that  the  computer  program  they  use  to  fill  out  the  DD350  form 
electronically  is  difficult  to  navigate,  and  the  program  often  rejects 
the  most  accurate  and  descriptive  information  because  the  data  en¬ 
tered  do  not  pass  certain  consistency  checks  that  are  not  transparent 
to  the  user.  In  addition,  many  contracting  officers  appeared  to  be  un¬ 
aware  that  the  DD350  data  are  now  being  used  for  important  analy¬ 
ses  related  to  purchasing  and  supply  management  strategies. 1  There¬ 
fore,  contracting  officers  may  be  more  concerned  with  getting  the 
DD350  system  to  accept  the  transaction  than  with  fine-tuning  the 
description  of  the  purchase. 


1  Several  interviewees  did  believe  that  the  Air  Force  and  small-business  community  do  pay 
attention  to  the  socioeconomic  data  reported  on  the  DD350  form  (e.g.,  size  of  a  company, 
ethnic  group  and  gender  of  business  owner). 
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Recommendations  to  Improve  the  Usefulness  of 
DD350  Data 

The  DD350  system  is  not  exclusively  an  Air  Force  system,  so  the  Air 
Force  does  not  control  the  design  of  the  form  or  the  software  used  to 
enter  data  into  the  DD350  database;  however,  we  believe  the  Air 
Force  can  take  several  steps  over  the  short  term  and  long  term  to  im¬ 
prove  the  usefulness  of  these  data  for  spend  analyses. 

Over  the  short  term,  it  would  be  helpful  to  communicate  to  the 
entire  contracting  workforce  that  these  data  are  now  being  used  to 
perform  detailed  analyses  to  support  implementation  of  new  pur¬ 
chasing  and  supply  management  strategies.  In  addition,  the  Air  Force 
could  collect  more  detailed  data  on  contracts  that  fall  within  the 
“problem”  PSC  and  dollar  categories  (identified  in  Chapter  Three)  to 
supplement  DD350  data  for  ongoing  spend  analyses.  With  enough 
additional  data,  the  Air  Force  may  be  able  to  use  statistical  analyses  to 
develop  guidelines  for  reallocating  dollars  among  activities  to  better 
reflect  the  realities  of  Air  Force  purchases.  For  example,  such  an 
analysis  might  suggest  that  x  percent  of  R&D  dollars  should  be  real¬ 
located  to  aircraft-related  purchase  categories. 

Over  the  long  term,  the  Air  Force  may  be  able  to  work  with 
other  branches  of  DoD  and  federal  agencies  to  refine  the  list  of  PSCs 
to  include  codes  that  better  describe  Air  Force  activities.  Potential 
new  codes  based  on  this  study  are  discussed  in  Chapter  Two  and 
listed  in  Appendix  B.2 

The  Air  Force  may  also  want  to  evaluate  the  costs  and  benefits 
of  being  able  to  describe  secondary  activities  in  more  detail.  Based  on 
our  analyses,  allowing  for  additional  PSCs  and  corresponding  dollar 
amounts  would  enhance  the  Air  Force’s  ability  to  describe  an  addi¬ 
tional  5  to  11  percent  of  DD350  contract  dollars,  which  equates  to 
$2  billion  to  $5  billion  for  FY02.  For  detailed  analyses  of  specific 


2  It  should  be  noted,  however,  that  creating  additional  codes  without  eliminating  other  less- 
useful  codes  could  introduce  the  possibility  of  increased  coding  errors.  Perhaps  the  Air  Force 
could  restrict  the  set  of  codes  its  contracting  officers  can  use  to  include  only  those  that  are 
most  appropriate  and  descriptive. 
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classes  of  goods  and/or  services,  this  additional  information  could  be 
quite  helpful  in  constructing  purchasing  strategies.  However,  lobby¬ 
ing  for  and  implementing  changes  to  the  DD350  form  and  the  addi¬ 
tional  training  those  changes  would  require  would  not  be  without 
costs.  In  addition,  entering  additional  details  into  the  DD350  form 
would  require  more  time  and  attention  from  contracting  officers  to 
avoid  the  possibility  of  increasing,  rather  than  decreasing,  coding 
problems. 

Finally,  the  Air  Force  may  benefit  from  providing  targeted  PSC 
coding  training  to  contracting  officers,  particularly  those  who  are 
working  with  technically  complex  contracts  or  contracts  that  include 
many  different  types  of  activities.  Such  training  should  include  spe¬ 
cific  guidance  for  resolving  the  types  of  coding  issues  discussed  in 
Chapter  Two. 


APPENDIX  A 


Individual  Contracting  Action  Report  Data  Fields 


Table  A.  1  presents  the  data  fields  on  the  October  2002  version  of  the 
Individual  Contracting  Action  Report  form,  also  known  as  the 
DD350  form. 

Table  A.1 

Sample  DD350  Form 


Data  Field 
Number 

Data  Field  Description 

Remarks 

A1 

Type  of  Report 

(0)  Original;  (1)  Canceling;  or  (2)  Correcting 

A2 

Report  Number 

A3 

Contracting  Office 

A3A 

A3A  Reporting 

Agency  FIPS  95 

Code 

A3B 

Contracting  Office 

Code 

A4 

Name  of  Contracting 
Office 

B1 

Contract  Identification 
Information 

B1 A 

Contract  Number 

BIB 

Origin  of  Contract 

(A)  DoD;  (B)  NASA;  or  (C)  Other  Non-DoD 
Agency 

B1C 

Bundled  Contract 

(Y)  Yes;  or  (N)  No 

BID 

Bundled  Contract 

(A)  Mission  Critical;  (B)  OMB  Circular  A-76; 

Exception 

or  (C)  Other 

B1E 

Performance-Based 
Service  Contract 

(Y)  Yes;  or  (N)  No 

B2 

Modification,  Order, 
or  Other  ID  Number 
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Table  A.1 — Continued 


Data  Field 
Number 

Data  Field  Description 

Remarks 

B3 

Action  Date 

(yyyymmdd) 

B4 

Completion  Date 

(yyyymmdd) 

B5 

Contractor 

Identification 

Information 

B5A 

Contractor 
Identification 
Number  (DUNS) 

B5B 

Government  Agency 

(Y)  Yes;  or  (N)  No 

B5C 

CAGE  Code 

B5D 

Contractor  Name  and 
Division  Name 

B5E 

Contractor  Address 

B5F 

Taxpayer 

Identification 

Number 

B5G 

Parent  Taxpayer 
Identification 
Number 

B5H 

Parent  Name 

B6 

Principal  Place  of 
Performance 

B6A 

City  or  Place  Code 

B6B 

State  or  Country  Code 

B6C 

City  or  Place  and 

State  or  Country 
Name 

B7 

Type  of  Obligation 

(1)  Obligation;  (2)  Deobligation;  or  (3)  No 
Dollars  Obligated  or  Deobligated 

B8 

Obligated  or 
Deobligated 

Dollars 

Enter  Whole  Dollars  Only 

B9 

Foreign  Military  Sale 

(Y)  Yes;  or  (N)  No 

BIO 

Multiyear  Contract 

(Y)  Yes;  or  (N)  No 

B11 

Total  Estimated 
Contract  Value 

Enter  Whole  Dollars  Only 

B 1 2 

Principal  Product  or 
Service 

B12A 

Federal  Supply  Class 
or  Service  Code 

B12B 

DoD  Claimant 

Program  Code 

B12C 

Program,  System,  or 
Equipment  Code 
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Data  Field 

Number  Data  Field  Description  Remarks 


B12D 

B 1 2  E 

NAICS  Code 

Name  or  Description 

B12F 

EPA-Designated 

Product(s) 

(A)  EPA-Designated  Product(s)  with 

Minimum  Recovered  Material  Content; 

(B)  FAR  23.405(c)(1)  Justification;  (C)  FAR 
23.405(c)(2)  Justification;  (D)  FAR 
23.405(c)(3)  Justification;  or  (E)  No  EPA- 
Designated  Product(s)  Acquired 

B12G 

B13 

Recovered  Material 
Clauses 

Kind  of  Action 

(A)  FAR  52.223-4;  or  (B)  FAR  52.223-4  and 
FAR  52.223-9 

B13A 

Contract  or  Order 

(1)  Letter  Contract;  (3)  Definitive  Contract; 
(4)  Order  under  an  Agreement;  (5)  Order 
under  Indefinite-Delivery  Contract; 

(6)  Order  under  Federal  Schedule;  (7)  BPA 
Order  under  Federal  Schedule;  (8)  Order 
from  UNICOR  or  JWOD;  or  (9)  Award 
under  FAR  Part  13 

B13B 

Type  of  Indefinite- 
Delivery  Contract 

(A)  Requirements  Contract  (FAR  52.216-21); 

(B)  Indefinite-Quantity  Contract  (FAR 
52.216-22);  or  (C)  Definite-Quantity 
Contract  (FAR  52.216-20) 

B13C 

Multiple  or  Single 
Award  Indefinite- 
Delivery  Contract 

(M)  Multiple  Award;  or  (S)  Single  Award 

B13D 

Modification 

(A)  Additional  Work  (new  agreement); 

(B)  Additional  Work  (other); 

(C)  Funding  Action;  (D)  Change  Order; 

(E)  Termination  for  Default; 

(F)  Termination  for  Convenience; 

(G)  Cancellation;  (H)  Exercise  of  an 

Option;  or  (J)  Definitization 

B13E 

Multiple  Award 
Contract  Fair 
Opportunity 

(A)  Fair  Opportunity  Process;  (B)  Urgency; 

(C)  One/Unique  Source;  (D)  Follow-On 
Contract;  or  (E)  Minimum  Guarantee 

B13F 

Indefinite-Delivery 
Contract  Use 

(A)  Government-Wide;  (B)  DoD-Wide; 

(C)  DoD  Department  or  Agency  Only;  or 

(D)  Contracting  Office  Only 

B13G 

Indefinite-Delivery 
Contract  Ordering 
Period  Ending  Date 

(yyyymmdd) 

B14 

CICA  Applicability 

(A)  Pre-CICA;  (B)  CICA  Applicable; 

(C)  Simplified  Acquisition  Procedures 
Other  than  FAR  Subpart  13.5;  or 

(D)  Simplified  Acquisition  Procedures 
Pursuant  to  FAR  Subpart  13.5 
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Table  A.1 — Continued 


Data  Field 
Number 

Data  Field  Description 

Remarks 

B15 

Information 

Technology 

Products  or  Services 

(A)  Commercially  Available  Off-the-Shelf 

Item;  (B)  Other  Commercial  Item  of 

Supply;  (C)  Nondevelopmental  Item 

Other  than  Commercial  Item;  (D)  Other 
Noncommercial  Item  of  Supply; 

(E)  Commercial  Service;  or 

(F)  Noncommercial  Service 

B 1 6 

Clinger-Cohen  Act 
Planning 

Compliance 

(Y)  Yes;  or  (N)  No 

Do  not  complete  Part  C  if  Line  B5B  is  coded  Y. 

Cl 

Synopsis 

(A)  Synopsis  Only;  (B)  Combined 
Synopsis/Solicitation;  or  (N)  Not 

Synopsized 

C2 

Reason  Not 

Synopsized 

(A)  Urgency;  (B)  FAR  5.202(a)(13); 

(C)  SBA/OFPP  Pilot  Program;  or  (Z)  Other 
Reason 

C3 

Extent  Competed 

(A)  Competed  Action;  (B)  Not  Available  for 
Competition;  (C)  Follow-On  to  Competed 
Action;  or  (D)  Not  Competed 

C4 

Sea  Transportation 

(Y)  Yes — Positive  Response  to  DFARS 
252.247-7022  or  252.212-7000(0(2); 

(N)  No — Negative  Response  to  DFARS 
252.247-7022  or  252.21 2-7000(c)(2);  or 
(U)  Unknown — No  Response  or  Provision 
Not  Included  in  Solicitation 

C5 

Type  of  Contract 

(A)  Fixed-Price  Redetermination;  (J)  Firm- 
Fixed-Price;  (K)  Fixed-Price  Economic  Price 
Adjustment;  (L)  Fixed-Price  Incentive; 

(M)  Fixed-Price  Award-Fee;  (R)  Cost-Plus- 
Award-Fee;  (S)  Cost  Contract;  (T)  Cost- 
Sharing;  (U)  Cost-Fixed-Fee;  (V)  Cost-Plus- 
Incentive-Fee;  (Y)  Time  and  Materials;  or 
(Z)  Labor  Hour 

C6 

Number  of  Offerors 
Solicited 

(1)  One;  or  (2)  More  than  One 

C7 

Number  of  Offers 
Received 
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Data  Field 

Number  Data  Field  Description  Remarks 


C8 

Solicitation 

(A)  Full  and  Open  Competition — Sealed  Bid; 

Procedures 

(B)  Full  and  Open  Competition — 
Competitive  Proposal;  (C)  Full  and  Open 
Competition — Combination;  (D)  Architect- 
Engineer;  (E)  Basic  Research;  (F)  Multiple 
Award  Schedule;  (G)  Alternative  Sources; 
(K)  Set-Aside;  or  (N)  Other  than  Full  and 
Open  Competition 

C9 

Authority  for  Other 

(1A)  Unique  Source;  (IB)  Follow-On  Contract; 

than  Full  and  Open 

(1C)  Unsolicited  Research  Proposal; 

Competition 

(ID)  Patent  or  Data  Rights;  (IE)  Utilities; 
(IF)  Standardization;  (1G)  Only  One 

Source — Other;  (2A)  Urgency; 

(3A)  Particular  Sources;  (4A)  International 
Agreement;  (5A)  Authorized  by  Statute; 
(5B)  Authorized  Resale;  (6A)  National 
Security;  or  (7A)  Public  Interest 

CIO 

Subject  to  Labor 

(A)  Walsh-Healey  Act;  (C)  Service  Contract 

Standards  Statutes 

Act;  (D)  Davis-Bacon  Act;  or  (Z)  Not 
Applicable 

C11 

Cost  or  Pricing  Data 

(Y)  Yes — Obtained;  (N)  No — Not  Obtained;  or 
(W)  Not  Obtained — Waived 

Cl  2 

Contract  Financing 

(A)  FAR  52.232-16;  (C)  Percentage  of 
Completion  Progress  Payments; 

(D)  Unusual  Progress  Payments  or  Advance 
Payments;  (E)  Commercial  Payments; 
Financing;  (F)  Performance-Based 

Financing;  or  (Z)  Not  Applicable 

C13 

Foreign  Trade  Data 

C13A 

Place  of  Manufacture 

(A)  U.S.;  or  (B)  Foreign 

C13B 

Country  of  Origin 

Code 

C14 

Commercial  Item 

(Y)  Yes — FAR  52.212-4  Included;  or 
(N)  No— FAR  52.212-4  Not  Included 

Do  not  complete  Part  D  if  Line  B5B  is  coded  Y  or  if  Line  B13A  is  coded  6. 

D1 

Type  of  Contractor 

D1 A 

Type  of  Entity 

(A)  Small  Disadvantaged  Business  (SDB) 
Performing  in  U.S.;  (B)  Other  Small 

Business  (SB)  Performing  in  U.S.;  (C)  Large 
Business  Performing  in  U.S.;  (D)  JWOD 
Participating  Nonprofit  Agency;  (F) 
Hospital;  (L)  Foreign  Concern  or  Entity; 

(M)  Domestic  Firm  Performing  Outside 
U.S.;  (T)  Historically  Black  College  or 
University  (HBCU);  (U)  Minority  Institution 
(Ml);  (V)  Other  Educational  or  (Z)  Other 
Nonprofit 
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A.1 — Continued 


Data  Field 

Number  Data  Field  Description  Remarks 


DIB 

Women-Owned 

Business 

(Y)  Yes;  or  (N)  No;  or  (U)  Uncertified 

D1C 

HUBZone 

Representation 

(Y)  Yes;  or  (N)  No 

DID 

Ethnic  Group 

(A)  Asian-lndian  American;  (B)  Asian-Pacific 
American;  (C)  Black  American;  (D) 

Hispanic  American;  (E)  Native  American; 

(F)  Other  SDB  Certified  or  Determined  by 
SBA;  or  (Z)  No  Representation 

DIE 

Veteran-Owned  Small 

(A)  Service-Disabled  Veteran;  or  (B)  Other 

Business 

Veteran 

D2 

Reason  Not  Awarded 

(A)  No  Known  SDB  Source;  (B)  SDB  Not 

to  SDB 

Solicited;  (C)  SDB  Solicited  and  No  Offer 
Received;  (D)  SDB  Solicited  and  Offer  Was 
Not  Low;  or  (Z)  Other  Reason 

D3 

Reason  Not  Awarded 

(A)  No  Known  SB  Source;  (B)  SB  Not 

to  SB 

Solicited;  (C)  SB  Solicited  and  No  Offer 
Received;  (D)  SB  Solicited  and  Offer  Was 
Not  Low;  or  (Z)  Other  Reason 

D4 

Set-Aside  or 

Preference 

Program 

D4A 

Type  of  Set-Aside 

(A)  None;  (B)  Total  SB  Set-Aside;  (C)  Partial 

SB  Set-Aside;  (D)  Section  8(a)  Set-Aside  or 
Sole  Source;  (E)  Total  SDB  Set-Aside; 

(F)  HBCU  or  Ml  Total  Set-Aside;  (G)  HBCU 
or  Ml — Partial  Set-Aside;  (H)  Very  Small 
Business  Set-Aside;  (J)  Emerging  Small 
Business  Set-Aside;  (K)  HUBZone  Set-Aside 
or  Sole  Source;  (L)  Combination  HUBZone 
and  8(a) 

D4B 

Type  of  Preference 

Type  of  Preference:  (A)  None;  (B)  SDB  Price 
Evaluation  Adjustment — Unrestricted; 

(C)  SDB  Preferential  Consideration — 
Partial  SB  Set-Aside;  (D)  HUBZone  Price 
Evaluation  Preference;  or 
(E)  Combination  HUBZone  Price 

Evaluation  Preference  and  SDB  Price 
Evaluation  Adjustment 

D4C 

Premium  Percent 

D7 

Small  Business 

(A)  Not  a  SBIR  Program  Phase  1,  II,  or  III; 

Innovation 

(B)  SBIR  Program  Phase  1  Action;  (C)  SBIR 

Research  (SBIR) 

Program  Phase  II  Action;  or  (D)  SBIR 

Program 

Program  Phase  III  Action 
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Data  Field 
Number 

Data  Field  Description 

Remarks 

D8 

Subcontracting 

(A)  Plan  Not  Included — No  Subcontracting 

Plan— SB,  SDB, 

Possibilities;  (B)  Plan  Not  Required; 

HBCU,  or  Ml 

(C)  Plan  Required — Incentive  Not 

Included;  or  (D)  Plan  Required — Incentive 
Included 

D9 

Small  Business 
Competitiveness 
Demonstration 
Program 

(Y)  Yes;  or  (N)  No 

DIO 

Size  of  Small  Business 

Employees  Annual  Gross  Revenues 

(A)  50  or  fewer  (M)  $1  million  or  less 

(B)  51-100  (N)  Over  $1  million- 

(C)  101-250  $2  million 

(D)  251-500  (p)  °ver  $2  million— 

(E)  501-750  $3.5  million 

(F)  751-1000  (R)  Over  $3.5  million- 

$5  million 

G  Over  1000  ...  _  .... 

(S)  Over  $5  million- 

$10  million 

(T)  Over  $10  million- 
$17  million 

Dll 

Emerging  Small 
Business 

(Y)  Yes;  or  (N)  No 

El 

Contingency, 

Humanitarian,  or 

Peacekeeping 

Operation 

(Y)  Yes;  or  Leave  Blank 

E2 

Cost  Accounting 
Standards  Clause 

(Y)  Yes;  or  Leave  Blank 

E3 

Requesting  Agency 
Code  (FIPS  95) 

E4 

Requesting  Activity 
Code 

E5 

Number  of  Actions 

E6 

Payment  by 

Governmentwide 
Purchase  Card 

(Y)  Yes;  or  Leave  Blank 

FI 

Name  of  Contracting 
Officer  or 
Representative 

F2 

Signature 

F3 

Telephone  Number 

APPENDIX  B 

Product  Service  Code  Categories  Used  in  the 
Analyses 


This  appendix  lists  the  PSC  categories  used  in  this  study  (see  Chapter 
Two  for  more  information).  Service  categories  (PSCs  that  begin  with 
a  letter  code)  are  listed  first,  followed  by  goods  categories  (PSCs  that 
begin  with  a  number  code).  The  boldface  lines  denote  a  single  PSC  or 
group  of  PSCs  that  was  used  in  the  analysis.  The  bulleted  lists  under 
those  lines  indicate  how  the  PSCs  were  subdivided. 


PSC  Service  Categories  Listed  by  First  Character  of  PSC 

A:  R&D 

•  AC:  RDT&E  for  Defense  Systems 

•  AD:  RDT&E  for  Defense — Other 

•  AR:  RDT&E  for  Space 

•  A  Other:  All  other  types  of  RDT&E  activities 

B:  Special  Studies  and  Analyses — Not  R&D 

•  B56:  Category  created  by  RAND  to  represent  well-defined  en¬ 
gineering  support  activities  that  typically  result  in  a  report  (e.g., 
on  how  to  address  obsolescence  problems  for  a  set  of  parts) 
and / or  a  set  of  data 

-  B560:  Aircraft 

-  B561:  Space 

•  B  Other:  All  other  PSCs  that  begin  with  the  letter  B.  These  are 
the  B  codes  found  in  the  official  PSC  list. 
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D:  Automatic  Data  Processing  (ADP)  and  Telecommunication  Services 

J:  Maintenance,  Repair,  and  Rebuilding  of  Equipment 

•  J 1 :  Category  created  by  RAND  to  represent  warranties.  The  last 
two  digits  of  this  code  are  the  first  two  digits  of  the  goods  codes 
to  which  the  warranty  applies 

•  J  Other:  All  other  PSCs  that  begin  with  the  letter  J.  These  are 
the  J  codes  found  in  the  official  PSC  list. 

K:  Modification  of  Equipment 

•  Kl:  Category  created  by  RAND  to  represent  engineering  sup¬ 
port  activities  associated  with  modifications 

•  K  Other:  All  other  PSCs  that  begin  with  the  letter  K.  These  are 
the  K  codes  found  in  the  official  PSC  list. 

L:  Technical  Representative  Services 

M:  Operation  of  Government-Owned  Facility 

N:  Installation  Equipment 

R:  Professional,  Administrative,  and  Management  Support  Services 

•  R4:  Professional  Services 

•  R  Other:  The  remainder  of  the  R  category  covering  Administra¬ 
tive  and  Management  Support  Services 

S:  Utilities  and  Housekeeping  Services 

•  SI:  Utilities 

•  S2:  Housekeeping  Services 

Other  Services 

•  C:  Architect  and  Engineering  Services — Construction 

•  E:  Purchase  of  Structures  and  Facilities 

•  F:  Natural  Resource  Management 

•  G:  Social  Services 
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•  H:  Quality  Control,  Testing,  and  Inspection  Services 

•  P:  Salvage  Services 

•  Q:  Medical  Services 

•  T:  Photographic,  Mapping,  Printing,  and  Publication  Services 

•  U:  Education  and  Training  Services 

•  V:  Transportation,  Travel,  and  Relocation  Services 

•  W:  Lease  or  Rental  of  Equipment 

•  X:  Lease  or  Rental  of  Facilities 

•  Y:  Construction  of  Structures  and  Facilities 

•  Z:  Maintenance,  Repair,  or  Alteration  of  Real  Property 


PSC  Goods  Categories  Listed  by  First  Two  Characters 
of  PSC 

“Aircraft  and  Components"  Category 

•  10:  Weapons 

•  11:  Nuclear  Ordnance 

•  12:  Fire  Control  Equipment 

•  13:  Ammunition  and  Explosives 

•  14:  Guided  Missiles 

•  15:  Aircraft  and  Airframe  Structural  Components 

•  16:  Aircraft  Components  and  Accessories 

•  17:  Aircraft  Launching,  Landing,  and  Ground  Handling 
Equipment 

•  18:  Space  Vehicles 

•  19:  Ships,  Small  Craft,  Pontoons,  and  Floating  Docks 

"Engines  and  Components"  Category 

•  20:  Ship  and  Marine  Equipment 

•  21:  Unassigned 

•  22:  Railway  Equipment 

•  23:  Ground  Effect  Vehicles 

•  24:  Tractors 

•  25:  Vehicular  Equipment  Components 

•  26:  Tires  and  Tubes 
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•  27:  Unassigned 

•  28:  Engines,  Turbines,  and  Components 

•  29:  Engine  Accessories 

Other  Goods 

•  All  other  goods  PSCs,  except  those  starting  with  the  numbers  1 
or  2. 


APPENDIX  C 

Survey  Questionnaire  for  Contracting  Personnel 


This  appendix  contains  the  questionnaire  we  used  to  gather  informa¬ 
tion  from  Air  Force  contracting  officers  about  the  contract  actions  in 
our  sample  of  FY02  DD350  actions.  We  used  a  subset  of  the  col¬ 
lected  information  for  the  analyses  described  in  this  report. 
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Hello,  this  is _ ,  and  I  am  calling  on  behalf  of  RAND,  a  non-profit 

research  institute.  I  sent  you  a  letter  recently  about  a  study  we  are  conducting 
for  General  Scott  in  SAF/AQC.  We  are  helping  SAF/AQC  better  understand 
what  the  Air  Force  buys,  who  it  buys  it  from,  and  how  purchasing  policies  and 
practices  might  be  improved.  As  part  of  this  process  we  are  examining 
purchases  recorded  in  the  DD350  database  and  would  like  to  ask  you  a  few 
question  about  the  two  contract  actions  identified  in  the  letter  I  sent  you. 

We  will  not  identify  your  name  or  the  individual  contract  actions  selected  for  our 
study.  We  will  also  send  you  copy  of  the  draft  report  when  it  is  available  this 
summer. 

Before  we  begin,  do  you  have  any  questions  about  the  study? 

1 .  Confirm  address  of  contracting  officer. 


Address  correct . di 

Address  incorrect . D2 

Address:  _ 


2.  Are  you  familiar  with  the  contract  actions  that  were  described  in  the  letter  I 
sent  you? 

Do  you  have  the  documentation  for  these  actions  available? 

I’d  like  to  start  by  verifying  the  information  we  have  about  the  first  contract 
action. 

[Ask  the  contracting  officer  for  each  of  the  following  pieces  information.  If 
different  from  what  is  in  DD350,  enter  new  data.] 


3.  Contracting  office 

Same  as  DD350 . di 

Different  than  DD350 . CI2 

Don’t  know . CJ3 


Survey  Questionnaire  for  Contracting  Personnel 
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4.  Total  amount  of  contract  action 


Same  as  DD350 . Di 

Different  than  DD350 . CJ2  $. 

Don’t  know . CJ3 

5.  Name  and  address  of  contractor 

Same  as  DD350 . Oi 

Different  than  DD350 . CJ2 

Name  of  contractor _ 

Contractor  address  _ 

Don’t  know  n3 

6.  Contractor’s  DUNS  number 

Same  as  DD350 .  Ei 

Different  than  DD350 .  ^2 

Don’t  know .  CI3 


7.  As  far  as  you  know,  is  this  contractor  part  of  a  larger  or  parent  company? 

Yes . □,  7a.  What  is  the  name  and  address  of  the  larger  or 

parent  company? 

Name  _ 

City  and  state 


Don't  know  C3i 

7b.  What  is  the  DUNs  number  of  the  larger  or  parent 
company? 

DUNs _ 

Don't  know . G3 

No .  n2 


Don’t  know....  CI3 
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8a.  What  is  the  start  date  for  this  contract  action? 


Same  as  DD350 . 

...□i 

Different  than  DD350... 

...□2 

fmm/yyl 

Don't  know . d3 

8b.  Is  this  start  date  for  the  contract  action  or  the  entire  contract? 


Contract  action . 

Entire  contract . 

...□2 

Don't  know  .  . 

n. 

What  is  the  end  date  or  completion  date  for  this  contract  action? 

Same  as  DD350 . 

Different  than  DD350... 

fmm/yvl 

Don't  know . 

...□3 

9b.  Is  this  end  date  for  the  contract  action  or  the  entire  contract? 

Contract  action . CJi 

Entire  contract . 02 

Don’t  know . Cl3 
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10.  Please  describe  the  different  types  of  services  (and  goods)  purchased  under  this  contract 
action. 


Description  of  Service  or  Good 

Service 

Good 

4-Digit 

PSC 

Amount 

10a. 

□, 

□2 

$ 

□2 

□i 

□2 

10b. 

a, 

□2 

$ 

□, 

□2 

a, 

□2 

10c. 

a, 

□2 

$ 

□. 

□2 

□2 

lOd. 

□2 

$ 

□t 

□2 

□i 

□2 

lOe. 

□< 

a2 

$ 

□i 

□2 

Percent  of 
Total  Amount 
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1 1 .  What  type  of  contract  action  is  this?  (E.g.,  fixed  price,  time  and  materials,  etc.) 


Fixed  price  redetermination . Gi 

Firm  fixed  price . CI2 

Fixed  price  econ  price  adjustment.. ..03 

Fixed  price  incentive . CJ4 

Fixed  price  award  fee . Os 

Cost  plus  award  fee . CU 

Cost  contract . CI7 

Cost  sharing . Os 

Cost  plus  fixed  fee . C]g 

Cost  plus  incentive  fee . Dio 

Time  and  materials . On 

Labor  hour . 

Other . Specify 


APPENDIX  D 

Characteristics  of  Contract  Actions  for  Which 
Interviews  Were  Completed 


Table  D.l  on  the  following  pages  describes  the  243  contract  actions 
for  which  interviews  with  contracting  officers  were  completed,  by 
dollar  size  of  the  contract  action  and  the  first  character  of  the  PSC  for 
the  contract  action  in  the  DD350  database.  Table  D.2  reports  the 
total  value  of  the  contract  actions  shown  in  each  cell  of  Table  D.l. 
The  analysis  in  this  report  is  based  on  these  243  contract  actions. 
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Table  D.1 

Contract  Actions  in  Analysis  Sample,  by  Dollar 


First  Character  of  PSC 
and  Type  of  Contract 
Action 

$0-$100K 

$100K- 

$500K 

Goods 

1  Aircraft  and 
components 

0 

1 

2  Engines  and 
components 

0 

1 

Other  Goods 

0 

2 

Total  Goods 

Services 

0 

4 

A  R&D 

3 

7 

B  Special  studies, 
not  R&D 

0 

0 

D  Automatic  data 
processing  and 
telecom  services 

1 

1 

J  Maintenance  and 
repair 

0 

6 

K  Modification  of 
equipment 

0 

0 

L  Technical 
representative 
services 

0 

0 

and  PSC 

Size  of  Contract  Action 


$500K- 

$1M 

$1M- 

$10M 

$10M- 

$50M 

$50M- 

$100M 

>  S100M 

Total 

1 

9 

2 

5 

18 

36 

1 

6 

2 

0 

1 

11 

1 

9 

3 

1 

0 

16 

3 

24 

7 

6 

19 

63 

8 

12 

17 

7 

9 

63 

0 

1 

0 

1 

0 

2 

4 

0 

2 

0 

0 

8 

1 

4 

2 

0 

0 

13 

1 

3 

2 

0 

0 

6 

0 

1 

1 

1 

2 

5 
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Table  D.1 — Continued 


Size  of  Contract  Action 

First  Character  of  PSC 
and  Type  of  Contract 
Action 

$0-$100K 

$100K- 

$500K 

$500K- 

$1M 

$1M- 

$10M 

$10M- 

$50M 

$50M- 

$100M 

>  $100M 

Total 

Services  (continued) 

M  Operation  of 
government- 
owned  facility 

0 

2 

0 

2 

1 

1 

0 

6 

R  Support  services 

1 

4 

6 

22 

7 

3 

0 

43 

S  Utilities  and 
housekeeping 

0 

0 

5 

7 

3 

0 

0 

15 

Other  Services 

0 

5 

6 

5 

1 

1 

1 

19 

Total  Services 

5 

25 

31 

57 

36 

14 

12 

180 

Total  Goods  and 
Services 

5 

29 

34 

81 

43 

20 

31 

243 
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Table  D.2 

Value  of  Contract  Actions  in  Analysis  Sample  ($  millions),  by  Dollar  Size  and  PSC 


Size  of  Contract  Action 


First  Character  of  PSC 
and  Type  of  Contract 

S100K- 

S500K- 

$1M- 

$10M- 

S50M- 

Action 

$0-$100K 

S500K 

$1M 

S10M 

$50M 

S100M 

>  $100M 

Total 

Goods 


1  Aircraft  and 
components 

0 

0 

1 

52 

52 

367 

8,570 

9,042 

2  Engines  and 
components 

0 

0 

1 

19 

36 

0 

308 

364 

Other  Goods 

0 

1 

1 

28 

64 

61 

0 

155 

Total  Goods 

Services 

0 

1 

3 

99 

152 

428 

8,878 

9,561 

A  R&D 

0 

2 

6 

50 

424 

424 

1,947 

2,853 

B  Special  studies, 
not  R&D 

0 

0 

0 

4 

0 

66 

0 

70 

D  Automatic  data 
processing  and 
telecom  services 

0 

0 

3 

0 

30 

0 

0 

33 

J  Maintenance  and 
repair 

0 

2 

1 

12 

31 

0 

0 

47 

K  Modification  of 
equipment 

0 

0 

1 

6 

29 

0 

0 

35 

L  Technical 
representative 

0 

0 

0 

2 

26 

75 

389 

491 

services 
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Table  D.2 — Continued 


Size  of  Contract  Action 

First  Character  of  PSC 
and  Type  of  Contract 
Action 

S0-S100K 

$100K- 

S500K 

S500K 

SIM 

$1M- 

$10M 

$10M- 

550M 

$50M- 

S100M 

>  S100M 

Total 

Services  (continued) 

M  Operation  of 
government- 
owned  facility 

0 

1 

0 

10 

21 

58 

0 

91 

R  Support  services 

0 

1 

5 

95 

167 

254 

0 

522 

S  Utilities  and 
housekeeping 

0 

0 

4 

26 

46 

0 

0 

76 

Other  Services 

0 

1 

4 

20 

17 

54 

115 

211 

Total  Services 

0 

8 

23 

225 

791 

931 

2,451 

4,429 

Total  Goods  and  0 

Services 

9 

26 

324 

943 

1,359 

10,944 

13,584 
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APPENDIX  E 


Extrapolating  Survey  Findings  to  All  Air  Force 
FY02  DD350  Contract  Actions 


This  appendix  describes  the  statistical  approach  we  used  to  extrapo¬ 
late  the  findings  from  our  survey  of  contracting  officers,  who  were 
responsible  for  the  contract  actions  in  our  sample,  to  the  entire  Air 
Force  FY02  DD350  database.  We  first  divided  the  sample  of  contract 
actions  into  14  separate  sampling  strata  made  up  of  two  activity-type 
categories  (goods  and  services),  each  of  which  has  seven  contract- 
action  dollar- amount  subcategories. 

The  seven  contract-action  dollar-amount  subcategories  are  as 
follows: 

•  <$100,000 

•  $100,000-$500,000 

•  $500,000-$  1  million 

•  $1  million-$10  million 

•  $10  million-$  50  million 

•  $50  million-$  100  million 

•  >  $100  million. 

We  then  reweighted  the  sample  results  for  each  stratum  to 
match  the  entire  population  of  Air  Force  FY02  DD350  actions  in 
one  of  two  ways — depending  on  whether  we  were  (1)  calculating  re¬ 
sults  with  respect  to  contract  actions  or  (2)  calculating  results  with 
respect  to  dollars  obligated  for  purchases. 

To  estimate  the  proportion  of  total  DD350  contract  actions  that 
have  a  particular  characteristic  (e.g.,  the  proportion  of  contract 
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actions  that  require  multiple  PSC  codes),  we  used  the  following 
equation: 

Pu>  ~  ^J^hPh 

where 

pjj  is  the  proportion  of  the  contract  actions  sampled  in  stratum 
h  with  the  particular  characteristic  \h  =  1 ...  14],  and  where 

W j}  is  the  proportion  of  all  contract  actions  in  the  DD350  data¬ 
base  that  are  in  stratum  h. 


Kish  (1995)  provides  the  following  formula  for  calculating  the 
variance  of  estimates  calculated  in  this  way: 


Var^)  =  SW?(1“//r) 


Ph^-Ph> 


nh 


-1 


where 

ri[:  is  the  sample  size  in  stratum  b,  and  where 

fh  is  the  ratio  of  sample  size  to  population  size  in  stratum  h. 

The  larger  the  sample  size  is  relative  to  the  population  in  each 
stratum  (/),),  the  closer  the  (1  —  jfy, )  term  is  to  zero  and  the  smaller  the 
variance  is.  Similarly,  the  further  from  50  percent  is  the  proportion 
of  actions  with  the  characteristic  of  interest  in  the  sample,  the  smaller 
the  Pifi-  py)  term  is  and  the  smaller  the  overall  variance  estimate  is. 
The  variance  is  then  used  to  calculate  the  confidence  interval. 

Table  E.l  provides  an  example  of  an  estimate  using  this  ap¬ 
proach.  This  calculation  underlies  the  result  reported  in  Table  3.5  in 
Chapter  Three — that  a  single  PSC  is  adequate  for  89  percent  of  con¬ 
tract  actions. 
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Table  E.1 

Numerical  Example  of  Method  Used  to  Estimate  Proportion  of  Contract 
Actions  with  a  Particular  Characteristic  (Percentage  of  Contract  Actions  for 
Which  One  PSC  Is  Adequate) 


Type/Dollar  Size  of  Contract 
Action 

1.  Proportion  of 
Contract  Actions 
in  Sample 
Requiring  Only 
One  PSC  ( ph ) 

2.  Proportion 
of  Total 
Population  of 
Contact  Actions 
That  Are  in 
Stratum  (Wh ) 

3.  Weighted 
Stratum 
Contribution  to 
Overall  Estimate 
(Column  1  x 
Column  2) 

Goods/<  $100K 

100% 

18.7% 

18.7% 

Goods/$100K-$500K 

100% 

8.4% 

8.4% 

Goods/$500K-$1M 

100% 

2.0% 

2.0% 

Goods/$1  M-$10M 

71% 

2.9% 

2.1% 

Goods/$10M-$50M 

57% 

0.3% 

0.2% 

Goods/$50M-$100M 

50% 

<0.1% 

<0.1% 

Goods/>  $100M 

84% 

<0.1% 

<0.1% 

Services/<  $100K 

80% 

35.3% 

28.2% 

Services/$100K-$500K 

92% 

22.0% 

20.2% 

Services/S  500K-$1M 

94% 

4.9% 

4.6% 

Services/$1  M-$10M 

86% 

5.0% 

4.3% 

Services/$10M-$50M 

58% 

0.4% 

0.2% 

Services/$50M-$100M 

50% 

<0.1% 

<0.1% 

Services/>  $100M 

67% 

<0.1% 

<0.1% 

Sum  ipw) 

88.9% 

When  extrapolating  sample  dollar  amounts  to  the  entire  DD350 
population,  we  use  the  combined-ratio  estimate  of  the  population 
share.  This  estimate  is  particularly  useful  when  the  sample  size  in  a 
stratum  is  small,  as  is  the  case  for  contract  actions  in  our  smaller- 
dollar  categories.  We  use  the  combined-ratio  estimate  ( yRc )  formula 
from  Cochran  (1977): 


A 


where 
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=  and  *,=5>*** 

/r  /r 

An  estimate  of  the  amount  of  expenditures  with  particular  char¬ 
acteristics  (e.g.,  the  expenditures  on  services)  is  formed  by  taking  the 
mean  of  the  expenditures  for  the  contract  actions  in  each  ac- 
tion/dollar-size  stratum  (j/;)  with  the  particular  characteristic  and 
multiplying  that  mean  by  the  number  of  actions  in  the  population  for 
each  stratum  ( N  ^ )  to  generate  an  expected  total  for  the  population 
in  each  stratum.  These  values  are  then  summed  across  categories  to 
yield  an  overall  total  for  expenditures  with  the  desired  characteristic 
(yj).  A  similar  estimate  is  calculated  using  the  mean  expenditures  for 
the  sample  for  the  entire  stratum  ( x ^ )  (with  and  without  the  desired 
characteristic),  multiplied  again  by  the  number  of  actions  in  the 
population  in  each  stratum  (TV^)  to  yield  a  total  for  the  entire  popu¬ 
lation  when  summed  across  categories  (Xj).  The  estimated  popula¬ 
tion  total  for  expenditures  with  the  desired  characteristic  ( y )  is  di¬ 
vided  into  the  estimated  population  of  all  expenditures  (jti)  to  arrive 
at  a  percentage  for  the  entire  population  ((y&.)/ X ).  This  method  al¬ 
lows  one  to  extrapolate,  for  example,  the  share  of  contract  dollars  in 
the  DD350  population  that  are  used  to  purchase  services. 

We  calculate  confidence  intervals  for  our  estimates  using  the 
variance  formula  provided  by  Cochran  (1977): 

r(4>  =  Y<N',{1- fh\s]h  +  R2sh,  -  2Rphs)hsxh) 

hnh 


where 

N jj  is  the  population  size  in  stratum  h, 
rifo  is  the  sample  size  in  stratum  h, 

fjj  is  the  ratio  of  sample  size  to  population  size  in  stratum  h, 
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2  * 

Syfj  is  the  variance  of  y  in  stratum  h, 

A 

R  is  the  estimated  share  (Y rc)  /  X  , 

S£j,  is  the  variance  of  x  in  stratum  h,  and 

Ph^ytjSxh 's  t^Le  covariance  of  x  andj y  in  stratum  h. 

Larger  sample  sizes  relative  to  the  population  in  each  stratum 
mean  that  (1 -/}_,)  in  the  first  term  of  the  variance  equation  will  be 
closer  to  zero,  reducing  the  variance  of  the  estimate.  Lower  variances 
in  the  dollar  values  in  the  sample  with  the  characteristic  of  interest 
and  for  total  expenditures  in  the  sample  also  lead  to  tighter  confi¬ 
dence  intervals. 

Table  E.2  illustrates  the  combined-ratio  estimate  calculation. 
The  table  yields  the  results  depicted  in  Figure  3.8 — that  spending  on 
services  represents  66  percent  of  total  Air  Force  spending  (the  sums  in 
the  last  row  of  the  table  are  divided  to  arrive  at  66  percent) . 

Reading  across  the  First  row  (goods  actions  for  less  than 
$100,000),  columns  1  through  3  are  interpreted  as  follows:  Column 
1  indicates  that  there  are  12,216  actions  in  the  Air  Force  FY02 
DD350  database  that  have  a  PSC  beginning  with  a  number  (indi¬ 
cating  that  they  are  goods  actions)  and  that  obligate  less  than 
$100,000  in  expenditures.  In  column  2,  for  actions  in  our  sample 
that  fall  within  this  category,  the  average  dollar  amount  that  we 
coded  as  services  is  $50,000.  In  column  3,  for  actions  in  our  sample 
that  fall  within  this  category  (whether  they  are  services  or  goods),  the 
average  action  size  is  $50,000.  Therefore,  all  of  the  goods  actions  in 
this  category  were  recoded  as  services  as  a  result  of  our  discussions 
with  the  relevant  contracting  officers  (see  Figure  3.3  in  Chapter 
Three). 


Table  E.2 

Numerical  Example  of  Combined  Ratio  Estimate 


Type  and  Size  of  Contract 
Action/ 

1.  Number  of 
Actions  in 
Population 

(Nh) 

2.  Mean  Dollar 
Value  of 
Actions  in 
Sample  with 
Characteristic 
($M)a 

Uh) 

3.  Total  Mean 
Dollar  Value  of 
Actions  in 
Sample  (SM) 

(H) 

4.  Estimated  Total 
Expenditures  with 
Characteristic  for 
Population  ($M) 
(Column  1  x 
Column  2)a 

(^i) 

5.  Estimated  Total 
Expenditures  for 
Population  ($M) 
(Column  1  x 
Column  3) 

(*l) 

Goods/<  $100K 

12,216 

0.05 

0.05 

583.0 

583.0 

Goods  /$100K-$500K 

5,477 

0.3 

0.3 

1,573.0 

1,573.0 

Goods/$500K-$1M 

1,279 

0.5 

0.7 

625.2 

946.0 

Goods/$1M-$10M 

1,923 

2.6 

5.0 

4,990.3 

9,692.2 

Goods/$10M-$50M 

225 

9.6 

21.6 

2,158.2 

4,860.0 

Goods/$50M-$100M 

24 

10.2 

71.4 

244.8 

1,713.6 

Goods/>  $100M 

21 

57.6 

469.0 

1,209.6 

9,849.0 

Services/<  $100K 

23,095 

0.1 

0.1 

1,346.1 

1,346.1 

Se  rvices/$  1 00  K-$  500  K 

14,378 

0.3 

0.3 

4,425.3 

4,425.3 

Services/S  500K-$1M 

3,226 

0.8 

0.8 

2,584.3 

2,670.2 

Services/S  1  M-$  1 0M 

3,296 

3.8 

4.0 

12,554.0 

13,110.1 

Services/S  10M-$50M 

279 

20.4 

22.0 

5,691.6 

6,138.0 

Services/$50M-$100M 

26 

38.6 

61.9 

1,003.6 

1,609.4 

Services/>  $100M 

13 

93.0 

204.0 

1,209.0 

2,652.0 

Sum 

40,198.0 

61,167.9 

a  The  characteristic  of  interest  here  is  whether  the  expenditures  were  for  services. 
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